-- Logs begin at Thu 2022-08-04 20:02:24 UTC, end at Thu 2022-08-04 21:45:16 UTC. -- Aug 04 21:01:39.547682 dv-cinder-852208-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 04 21:01:39.559609 dv-cinder-852208-1-iscsi devstack@keystone.service[80588]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 04 21:01:39.561605 dv-cinder-852208-1-iscsi devstack@keystone.service[80588]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Aug 4 21:01:39 2022] *** Aug 04 21:01:39.561605 dv-cinder-852208-1-iscsi devstack@keystone.service[80588]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 04 21:01:39.561605 dv-cinder-852208-1-iscsi devstack@keystone.service[80588]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 04 21:01:39.561605 dv-cinder-852208-1-iscsi devstack@keystone.service[80588]: nodename: dv-cinder-852208-1-iscsi Aug 04 21:01:39.561605 dv-cinder-852208-1-iscsi devstack@keystone.service[80588]: machine: x86_64 Aug 04 21:01:39.561605 dv-cinder-852208-1-iscsi devstack@keystone.service[80588]: clock source: unix Aug 04 21:01:39.561605 dv-cinder-852208-1-iscsi devstack@keystone.service[80588]: pcre jit disabled Aug 04 21:01:39.561605 dv-cinder-852208-1-iscsi devstack@keystone.service[80588]: detected number of CPU cores: 4 Aug 04 21:01:39.561605 dv-cinder-852208-1-iscsi devstack@keystone.service[80588]: current working directory: / Aug 04 21:01:39.561605 dv-cinder-852208-1-iscsi devstack@keystone.service[80588]: detected binary path: /usr/bin/uwsgi-core Aug 04 21:01:39.562254 dv-cinder-852208-1-iscsi devstack@keystone.service[80588]: your processes number limit is 47680 Aug 04 21:01:39.562254 dv-cinder-852208-1-iscsi devstack@keystone.service[80588]: your memory page size is 4096 bytes Aug 04 21:01:39.562254 dv-cinder-852208-1-iscsi devstack@keystone.service[80588]: detected max file descriptor number: 2048 Aug 04 21:01:39.562254 dv-cinder-852208-1-iscsi devstack@keystone.service[80588]: lock engine: pthread robust mutexes Aug 04 21:01:39.562254 dv-cinder-852208-1-iscsi devstack@keystone.service[80588]: thunder lock: enabled Aug 04 21:01:39.562254 dv-cinder-852208-1-iscsi devstack@keystone.service[80588]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 04 21:01:39.562254 dv-cinder-852208-1-iscsi devstack@keystone.service[80588]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 04 21:01:39.602155 dv-cinder-852208-1-iscsi devstack@keystone.service[80588]: Python main interpreter initialized at 0x5581fbce39c0 Aug 04 21:01:39.602155 dv-cinder-852208-1-iscsi devstack@keystone.service[80588]: python threads support enabled Aug 04 21:01:39.602155 dv-cinder-852208-1-iscsi devstack@keystone.service[80588]: your server socket listen backlog is limited to 100 connections Aug 04 21:01:39.602155 dv-cinder-852208-1-iscsi devstack@keystone.service[80588]: your mercy for graceful operations on workers is 90 seconds Aug 04 21:01:39.602471 dv-cinder-852208-1-iscsi devstack@keystone.service[80588]: mapped 403077 bytes (393 KB) for 2 cores Aug 04 21:01:39.602569 dv-cinder-852208-1-iscsi devstack@keystone.service[80588]: *** Operational MODE: preforking *** Aug 04 21:01:39.602841 dv-cinder-852208-1-iscsi devstack@keystone.service[80588]: *** uWSGI is running in multiple interpreter mode *** Aug 04 21:01:39.602841 dv-cinder-852208-1-iscsi devstack@keystone.service[80588]: spawned uWSGI master process (pid: 80588) Aug 04 21:01:39.603032 dv-cinder-852208-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 04 21:01:39.604161 dv-cinder-852208-1-iscsi devstack@keystone.service[80588]: spawned uWSGI worker 1 (pid: 80600, cores: 1) Aug 04 21:01:39.604161 dv-cinder-852208-1-iscsi devstack@keystone.service[80588]: spawned uWSGI worker 2 (pid: 80601, cores: 1) Aug 04 21:01:39.604161 dv-cinder-852208-1-iscsi devstack@keystone.service[80588]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 04 21:01:45.179150 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 04 21:01:45.179150 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: from cryptography.utils import int_from_bytes Aug 04 21:01:45.180005 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 04 21:01:45.180005 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: from cryptography.utils import int_from_bytes Aug 04 21:01:45.180502 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 04 21:01:45.180502 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: from cryptography.utils import int_from_bytes Aug 04 21:01:45.180956 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 04 21:01:45.180956 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: from cryptography.utils import int_from_bytes Aug 04 21:01:45.230274 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 04 21:01:45.230477 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 04 21:01:45.230674 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] command line args: [] {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 04 21:01:45.230954 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 04 21:01:45.231135 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ================================================================================ {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 04 21:01:45.231339 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 04 21:01:45.231427 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] admin_token = **** {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.231544 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 04 21:01:45.231651 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] config_dir = [] {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.231730 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] command line args: [] {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 04 21:01:45.231844 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.231994 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 04 21:01:45.232079 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] config_source = [] {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.232171 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ================================================================================ {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 04 21:01:45.232255 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] debug = True {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.232383 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.232660 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] admin_token = **** {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.232660 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] config_dir = [] {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.232775 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.232838 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.232979 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.233063 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] config_source = [] {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.233169 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] debug = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.233241 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.233355 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.233462 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.233584 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.233671 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] list_limit = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.233773 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.233851 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] log_config_append = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.233973 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.234057 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.234160 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] log_dir = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.234241 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.234344 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] log_file = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.234458 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] list_limit = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.234565 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.234637 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] log_config_append = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.234803 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.234896 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.234975 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] log_dir = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.235051 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.235150 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] log_file = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.235296 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.235406 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.235516 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.235663 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.235812 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.235926 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.236023 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.236143 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.236257 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.236336 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.236431 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.236506 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.236584 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.236711 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.236843 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.236957 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.237039 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.237161 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.237291 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.237380 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.237540 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.237616 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.237821 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.237903 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.238064 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.238309 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.238432 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.25/identity {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.238519 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] publish_errors = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.238603 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.238747 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.238980 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.239113 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.25/identity {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.239200 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.239349 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] publish_errors = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.239425 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.239542 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.239658 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.239760 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.239895 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] standard_threads = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.239981 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.240129 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.240216 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.240376 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.240481 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.240560 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.240637 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] standard_threads = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.240707 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] use_journal = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.240823 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.240902 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] use_json = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.241047 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] use_stderr = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.241124 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.241225 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] use_syslog = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.241303 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.241397 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] use_journal = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.241478 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.241560 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] use_json = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.241702 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.241785 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] use_stderr = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.241894 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] use_syslog = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.241976 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.242121 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:01:45.242275 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.242376 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.242455 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.242613 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.242696 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.242871 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.242959 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.243110 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.243187 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.243365 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.243445 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.243604 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.243692 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.243850 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.243933 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.244085 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.244169 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.244323 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.244407 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.244565 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.244641 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.244801 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.244876 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.245036 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.245113 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.245280 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.245358 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.245513 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.245599 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.245757 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.245835 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.246003 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.246089 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.246239 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.246323 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.246475 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.246552 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.246730 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.246811 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.246974 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.247050 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.247207 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.247285 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.247487 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.247565 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.247723 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.247799 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.247952 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.248033 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.248179 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.248262 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.248417 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.248494 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.248651 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.248734 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.248904 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.248986 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.249136 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.249213 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.249371 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.249449 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.249626 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.249703 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.249858 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.249934 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.250088 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.250167 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.250334 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.250410 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.250564 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.250639 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.250803 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.250882 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.251040 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.251144 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.251267 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.251422 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.251530 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.251626 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.251741 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.251845 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.251969 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.252100 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.252223 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.252305 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.252427 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.252503 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.252617 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.252699 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.252842 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.252927 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.253091 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.253169 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.253322 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.253411 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.253558 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.253636 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.253785 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.253872 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.254019 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.254107 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.254792 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.254880 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.255039 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.255124 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.255283 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.255399 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.255551 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.255680 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.255793 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.255871 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.256029 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.256109 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.256266 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.256347 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.256504 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.256584 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.256735 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.256816 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.256968 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.257048 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.257207 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.257285 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.257444 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.257528 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.257679 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.257766 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.257917 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.257995 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.258158 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.258235 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.258384 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.258469 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.258623 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.258701 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.258860 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.258937 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.259125 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.259211 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.259388 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.259490 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.259633 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.259730 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.259874 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.260029 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.260104 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.260173 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.260331 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.260409 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.260567 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.260653 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.260819 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.260900 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.261057 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.261140 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.261292 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.261381 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.261527 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.261606 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.261761 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.261837 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.261998 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.262075 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.262230 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.262309 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.262466 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.262548 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.262704 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.262782 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.262950 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.263029 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.263191 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.263274 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.263454 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.263532 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.263693 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.263772 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.263928 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.264005 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.264163 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.264239 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.264393 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.264471 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.264627 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.264704 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.264859 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.264937 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.265090 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.265174 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.265318 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.265394 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.265559 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.265636 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.265789 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.265869 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.266030 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.266107 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.266266 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.266351 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.266518 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.266597 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.266755 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.266833 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.266991 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] auth.external = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.267067 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.267226 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.267306 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.267523 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.267601 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.267757 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] auth.external = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.267835 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.267988 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.268076 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] auth.password = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.268218 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] auth.token = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.268297 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.268460 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.268539 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.268699 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.268777 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] auth.password = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.268938 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] auth.token = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.269017 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.269170 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.269248 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.269401 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.269478 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.269646 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.269760 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.270455 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.270551 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.270704 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] credential.caching = True {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.270814 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.270956 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.271038 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.271217 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.271377 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.271457 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] credential.caching = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.271541 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.271704 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.271784 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.271951 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.272029 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.272189 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.272279 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.272424 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.272503 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.272669 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.272755 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.272904 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.272991 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.273166 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.273246 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.273421 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.273508 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.273660 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.273747 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.273913 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.273993 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.274170 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.274248 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.274425 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.274580 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.274663 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] federation.caching = True {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.274742 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.274887 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.274966 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.275136 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.275224 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.275398 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] federation.caching = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.275480 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.275635 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.275713 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.275879 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.275969 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.276117 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.276196 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.276343 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.276421 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.276590 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.276677 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.276828 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.276908 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.277073 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.277151 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.277307 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.277394 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.277550 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] identity.caching = True {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.277634 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.277784 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.277862 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.278031 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.278118 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.278266 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] identity.caching = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.278346 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.278502 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.278579 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.278743 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.278828 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.278976 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.279054 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.279206 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.279283 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.279480 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.279571 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.279714 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.279790 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.279931 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.280010 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.280180 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.280268 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.280420 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.280505 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.280651 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.280728 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.280882 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.280969 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.281114 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.281192 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.281357 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.281435 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.281604 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.281685 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.281849 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.281928 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.282105 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.282184 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.282347 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.282426 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.282837 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.282922 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.283090 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.283168 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.283333 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.283415 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.283578 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.283658 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.283818 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.283896 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.284051 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.284132 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.284294 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.284373 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.284527 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.284610 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.284763 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.284843 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.285012 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.285090 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.285242 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.285320 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.285519 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.285599 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.285771 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.285849 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.286006 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.286084 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.286248 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.286329 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.286484 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.286572 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.286710 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.286788 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.286951 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.287031 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.287182 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.287261 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.287443 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.287523 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.287680 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.287766 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.287923 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.288002 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.288161 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.288240 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.288393 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.288472 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.288627 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.288751 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.288871 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.288949 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.289102 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.289183 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.289332 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.289410 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.289571 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.289647 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.289811 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.289892 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.290042 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.290137 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.user = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.290278 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.290357 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.290519 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.290600 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.290750 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.user = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.290829 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.290989 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.291072 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.291237 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.291375 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.291494 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.291573 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.291737 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.291815 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.291973 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.292054 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.292213 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.292292 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.292443 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.292521 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.292673 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.292791 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.292903 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.292982 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.293132 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.293211 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.293368 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.293448 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.293599 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.293678 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.293847 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.293964 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.294093 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.294218 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.294339 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.294455 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.294585 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.294708 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.294829 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.294951 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.295059 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.295193 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.295305 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.295475 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.295704 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.295783 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.295951 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.296030 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.296189 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.296278 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.296429 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.296534 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.296688 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.296767 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.296923 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.297009 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.297172 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.297252 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.297413 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.297492 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.297653 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.297803 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.297879 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.297951 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.298133 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.298212 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.298375 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.298461 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.298607 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.298685 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.298848 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.298927 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.299086 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.299175 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] resource.caching = True {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.299385 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.299481 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.299578 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.299658 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.299818 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] resource.caching = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.299904 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.300070 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.300152 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.300317 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.300396 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.300559 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.300644 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.300806 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.300884 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.301071 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.301152 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.301304 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.301390 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.301540 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] role.caching = True {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.301618 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.301793 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] role.driver = sql {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.301871 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.302028 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.302115 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.302281 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.302360 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] role.caching = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.302533 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] role.driver = sql {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.302611 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.302766 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.302853 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.303016 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.303096 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.303263 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.303361 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.303539 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.303630 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.303786 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.303865 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.304034 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.304111 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.304269 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.304357 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.304512 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.304591 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.304758 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.304837 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.304989 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.305079 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.305236 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.305315 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.305483 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.305647 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.305735 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.305889 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.305968 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.306146 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.306336 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.306415 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.306573 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.306653 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.306820 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.306900 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.307069 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.307149 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.307351 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.307435 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.307585 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.307665 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.307821 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.307901 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.308069 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.308156 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.308305 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.308384 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.308540 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.308618 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.308768 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.308848 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.308998 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.309077 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.309230 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.309308 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.309480 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.309561 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.309730 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.309808 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.309964 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.310043 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.310199 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.310278 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] token.caching = True {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.310437 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.310516 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.310667 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.310746 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.310902 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] token.caching = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.310982 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.311138 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.311220 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.311426 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.311507 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.311670 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.311753 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.311920 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.311999 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.312168 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.312247 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.312408 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.312487 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.312649 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.312727 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.312887 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.312965 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.313122 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.313201 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.313356 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.313433 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.313594 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.313673 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.313828 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.313907 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.314071 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.314149 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.314318 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.314411 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.314548 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.314627 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] database.connection = **** {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.314788 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.314876 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.315080 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.315160 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.315363 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.315502 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] database.connection = **** {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.315590 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.315671 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.315810 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.315897 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.316048 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.316128 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.316287 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.316367 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.316518 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.316596 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.317271 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.317352 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.317498 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.317575 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.317728 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.317808 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.317972 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.318050 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.318209 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.318290 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.318446 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.318524 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.318678 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.318756 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.318917 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.318996 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.319154 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.319231 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.319421 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.319506 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.319590 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 04 21:01:45.319664 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.319880 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.320115 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:01:45.320286 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 04 21:01:45.349064 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:01:45.349296 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:01:45.349528 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:01:45.349611 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:01:45.349760 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:01:45.349986 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:01:45.357922 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:01:45.358317 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:01:45.388774 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:01:45.389007 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:01:45.389332 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:01:45.389576 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:01:45.442603 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:01:45.442809 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:01:45.493050 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:01:45.493283 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:01:45.494140 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:01:45.494380 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:01:45.558029 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:01:45.558255 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:01:45.558343 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:01:45.558480 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:01:45.558601 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:01:45.558729 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:01:45.558812 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:01:45.558950 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:01:45.559032 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:01:45.559170 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:01:45.559253 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:01:45.559413 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:01:45.582169 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:01:45.582896 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:01:45.583000 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:01:45.583134 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:01:45.583217 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:01:45.583388 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:01:45.583471 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:01:45.583619 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:01:45.583714 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:01:45.583838 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:01:45.583919 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:01:45.584056 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:01:45.584137 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:01:45.584275 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:01:45.595094 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:01:45.595364 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:01:45.607868 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:01:45.608142 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:01:45.608241 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:01:45.608392 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:01:45.608475 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:01:45.608618 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:01:45.608708 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:01:45.608833 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:01:45.608925 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:01:45.609048 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:01:45.609129 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:01:45.609260 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:01:45.609339 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:01:45.609474 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:01:45.628269 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.628479 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 04 21:01:45.628818 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.628903 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.629023 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 04 21:01:45.629160 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 04 21:01:45.629372 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.630049 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.630133 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.630260 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 04 21:01:45.630391 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.630534 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.630616 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.630791 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.631046 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.631301 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.640414 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.640693 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.640966 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.641049 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.641238 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.641335 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.641508 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.641791 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.648251 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80601) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 21:01:45.648740 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80600) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 21:01:45.653237 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80601) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 21:01:45.653435 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80600) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 21:01:45.653519 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.653733 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.653823 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.653987 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.654071 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.654263 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.654347 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.654543 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.654625 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.654801 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.654886 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.655130 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.655210 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.655433 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.655529 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.655707 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.655788 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.655980 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.656061 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.656247 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.680097 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.680192 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.681590 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80600) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 21:01:45.681680 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80601) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 21:01:45.681869 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.681965 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.686527 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80600) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 21:01:45.686616 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80601) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 21:01:45.686805 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.686889 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.687087 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.687174 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.694607 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80600) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 21:01:45.694705 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80601) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 21:01:45.694843 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.694936 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.698923 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80600) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 21:01:45.699010 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80601) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 21:01:45.699210 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.699292 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.699517 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.699639 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.699800 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.699883 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.700088 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.700171 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.700412 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.700495 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.700687 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.700768 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.700962 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.701054 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.716822 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.717027 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.717111 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.717310 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.720981 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80600) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 21:01:45.721267 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80601) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 21:01:45.724198 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.724492 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.724575 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.724791 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.728876 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80600) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 21:01:45.729237 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80601) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 21:01:45.732282 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80600) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 21:01:45.732652 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80601) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 21:01:45.735455 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.735752 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.735835 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.736037 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.736119 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.736322 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.736427 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.736609 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.736699 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.736896 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.736979 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.737233 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.753305 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80600) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 21:01:45.753587 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.753788 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80601) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 21:01:45.753883 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.754075 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.754185 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.754410 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.754681 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.761005 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.761578 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.762554 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.762840 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.763140 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.763451 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.765685 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80600) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 21:01:45.765967 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.766256 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.766346 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80601) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 21:01:45.766539 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.766638 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.766867 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.766959 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.767196 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.767513 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.778240 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80600) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 21:01:45.778521 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.778802 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.778967 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80601) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 21:01:45.779086 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.779250 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.779374 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.779561 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.779647 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.779846 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.779969 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.780127 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.780411 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.780695 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.796250 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80600) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 21:01:45.797286 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80601) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 21:01:45.799185 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80600) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 21:01:45.800297 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80601) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 21:01:45.802172 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.803293 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.804680 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.805776 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.806171 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80600) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 21:01:45.806456 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.806744 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.807266 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80601) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 21:01:45.807578 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.807871 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.813169 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.814312 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.814764 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80600) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 21:01:45.815964 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80601) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 21:01:45.818228 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.818513 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.818796 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.819067 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.819463 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.819759 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.820049 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.822376 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.827404 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80600) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 21:01:45.827692 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.827971 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.833737 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80601) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 21:01:45.834026 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.834313 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.835732 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80600) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 21:01:45.836009 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.836301 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.836573 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.836844 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.837113 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.837383 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.837669 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.838047 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.838320 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.838602 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.839045 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.839309 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.839598 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.841236 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80601) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 21:01:45.841522 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.841815 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.842098 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.842377 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.842655 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.842933 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.843225 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.843639 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.843919 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.844207 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.844475 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.844748 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.845022 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:01:45.872640 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5581fbce39c0 pid: 80600 (default app) Aug 04 21:01:45.876104 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9cdf27-dce4-4c63-bcc2-89abd12349f5 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:45.876393 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9cdf27-dce4-4c63-bcc2-89abd12349f5 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:45.876664 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9cdf27-dce4-4c63-bcc2-89abd12349f5 None None] PATH_INFO: `/v3` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:45.878439 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5581fbce39c0 pid: 80601 (default app) Aug 04 21:01:45.881583 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1/1] 10.222.0.25 () {50 vars in 791 bytes} [Thu Aug 4 21:01:45 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:01:54.506804 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b699c7e6-bb5a-4f3b-9378-72aed37c1783 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:54.507390 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b699c7e6-bb5a-4f3b-9378-72aed37c1783 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:54.507390 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b699c7e6-bb5a-4f3b-9378-72aed37c1783 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:54.511753 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1/2] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:01:54 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:01:54.516939 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa2d735-56bd-4155-98cd-deae36f11877 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:54.517355 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa2d735-56bd-4155-98cd-deae36f11877 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:54.517489 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa2d735-56bd-4155-98cd-deae36f11877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:54.588123 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: INFO dbcounter [None req-1aa2d735-56bd-4155-98cd-deae36f11877 None None] Registered counter for database keystone Aug 04 21:01:54.607785 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG oslo_db.sqlalchemy.engines [None req-1aa2d735-56bd-4155-98cd-deae36f11877 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80600) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 04 21:01:54.614252 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writer thread running {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 04 21:01:54.792271 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG passlib.handlers.bcrypt [None req-1aa2d735-56bd-4155-98cd-deae36f11877 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80600) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 04 21:01:54.792271 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG passlib.handlers.bcrypt [None req-1aa2d735-56bd-4155-98cd-deae36f11877 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80600) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 04 21:01:54.844985 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-1aa2d735-56bd-4155-98cd-deae36f11877 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:01:54.846578 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-1aa2d735-56bd-4155-98cd-deae36f11877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:01:54.975503 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 2/3] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:01:54 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 459 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 04 21:01:55.131814 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b699c7e6-bb5a-4f3b-9378-72aed37c1783 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:55.202593 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: INFO dbcounter [None req-b699c7e6-bb5a-4f3b-9378-72aed37c1783 None None] Registered counter for database keystone Aug 04 21:01:55.223950 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG oslo_db.sqlalchemy.engines [None req-b699c7e6-bb5a-4f3b-9378-72aed37c1783 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80601) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 04 21:01:55.230303 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writer thread running {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 04 21:01:55.350315 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b699c7e6-bb5a-4f3b-9378-72aed37c1783 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:55.354748 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c6c553-3f34-4f67-8676-2580709f8ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:55.355412 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c6c553-3f34-4f67-8676-2580709f8ffb admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:55.355700 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c6c553-3f34-4f67-8676-2580709f8ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:55.355979 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c6c553-3f34-4f67-8676-2580709f8ffb admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:55.365034 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c6c553-3f34-4f67-8676-2580709f8ffb admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:55.417216 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: WARNING keystone.common.rbac_enforcer.enforcer [None req-75c6c553-3f34-4f67-8676-2580709f8ffb admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 04 21:01:55.417554 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c6c553-3f34-4f67-8676-2580709f8ffb admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:55.427805 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 2/4] 10.222.0.25 () {58 vars in 1324 bytes} [Thu Aug 4 21:01:55 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 21:01:55.433872 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa2d735-56bd-4155-98cd-deae36f11877 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:55.449195 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa2d735-56bd-4155-98cd-deae36f11877 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:55.453682 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126b53d9-722a-4235-a848-8f255e1e8199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:55.454790 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b53d9-722a-4235-a848-8f255e1e8199 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:55.455067 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b53d9-722a-4235-a848-8f255e1e8199 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:55.455356 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b53d9-722a-4235-a848-8f255e1e8199 admin admin] PATH_INFO: `/v3/projects/6da47929012341928c673a2e0d1e97f0` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:55.456835 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126b53d9-722a-4235-a848-8f255e1e8199 admin admin] RBAC: Authorizing `identity:get_project(project_id=6da47929012341928c673a2e0d1e97f0)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:55.508264 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: WARNING keystone.common.rbac_enforcer.enforcer [None req-126b53d9-722a-4235-a848-8f255e1e8199 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 04 21:01:55.508603 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126b53d9-722a-4235-a848-8f255e1e8199 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:55.513388 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 3/5] 10.222.0.25 () {56 vars in 1207 bytes} [Thu Aug 4 21:01:55 2022] GET /identity/v3/projects/6da47929012341928c673a2e0d1e97f0 => generated 344 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:01:57.475458 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0079d04-c57c-42c6-a6b9-cb5a91503951 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:57.475953 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0079d04-c57c-42c6-a6b9-cb5a91503951 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:57.475953 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0079d04-c57c-42c6-a6b9-cb5a91503951 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:57.480059 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 3/6] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:01:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:01:57.485493 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d00299-f91d-43cd-8872-6e9f216fb4e5 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:57.485888 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d00299-f91d-43cd-8872-6e9f216fb4e5 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:57.486051 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d00299-f91d-43cd-8872-6e9f216fb4e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:57.505604 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-a0d00299-f91d-43cd-8872-6e9f216fb4e5 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:01:57.507121 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-a0d00299-f91d-43cd-8872-6e9f216fb4e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:01:57.515040 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 4/7] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:01:57 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 04 21:01:57.538041 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0079d04-c57c-42c6-a6b9-cb5a91503951 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:57.549853 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0079d04-c57c-42c6-a6b9-cb5a91503951 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:57.554116 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9aef11-d5c6-4780-aec9-316d04de826a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:57.554748 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9aef11-d5c6-4780-aec9-316d04de826a admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:57.555028 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9aef11-d5c6-4780-aec9-316d04de826a admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:57.555303 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9aef11-d5c6-4780-aec9-316d04de826a admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:57.562834 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9aef11-d5c6-4780-aec9-316d04de826a admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:57.564639 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9aef11-d5c6-4780-aec9-316d04de826a admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:57.573223 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 4/8] 10.222.0.25 () {58 vars in 1324 bytes} [Thu Aug 4 21:01:57 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 21:01:57.578494 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d00299-f91d-43cd-8872-6e9f216fb4e5 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:57.587948 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d00299-f91d-43cd-8872-6e9f216fb4e5 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:57.592306 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea75df6-cb13-41a3-b0e8-5e1ab5447d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:57.592938 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea75df6-cb13-41a3-b0e8-5e1ab5447d72 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:57.593224 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea75df6-cb13-41a3-b0e8-5e1ab5447d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:57.593502 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea75df6-cb13-41a3-b0e8-5e1ab5447d72 admin admin] PATH_INFO: `/v3/users/bc6bad33dfb24427b1ccf30de722b45a` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:57.594853 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea75df6-cb13-41a3-b0e8-5e1ab5447d72 admin admin] RBAC: Authorizing `identity:get_user(user_id=bc6bad33dfb24427b1ccf30de722b45a)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:57.596761 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea75df6-cb13-41a3-b0e8-5e1ab5447d72 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:57.598900 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 5/9] 10.222.0.25 () {56 vars in 1198 bytes} [Thu Aug 4 21:01:57 2022] GET /identity/v3/users/bc6bad33dfb24427b1ccf30de722b45a => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:01:59.659009 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d20bfc0-ce7e-4459-a600-a30497fbe2bf None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:59.659550 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d20bfc0-ce7e-4459-a600-a30497fbe2bf None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:59.659643 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d20bfc0-ce7e-4459-a600-a30497fbe2bf None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:59.660994 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 5/10] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:01:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:01:59.666057 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992073de-37f0-4364-b0e0-88ebbdd09783 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:59.666469 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992073de-37f0-4364-b0e0-88ebbdd09783 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:59.666626 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992073de-37f0-4364-b0e0-88ebbdd09783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:59.686092 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-992073de-37f0-4364-b0e0-88ebbdd09783 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:01:59.687282 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-992073de-37f0-4364-b0e0-88ebbdd09783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:01:59.692371 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e0e0b5-bc5a-4948-865c-d2723edcdbdf None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:59.692650 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e0e0b5-bc5a-4948-865c-d2723edcdbdf None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:59.692921 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e0e0b5-bc5a-4948-865c-d2723edcdbdf None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:59.694238 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 6/11] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:01:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 21:01:59.700117 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20b6f16-39c8-489e-bc36-7571e2ddc92d None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:59.700406 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20b6f16-39c8-489e-bc36-7571e2ddc92d None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:59.700681 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20b6f16-39c8-489e-bc36-7571e2ddc92d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:59.743990 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG passlib.handlers.bcrypt [None req-c20b6f16-39c8-489e-bc36-7571e2ddc92d None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80601) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 04 21:01:59.744412 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG passlib.handlers.bcrypt [None req-c20b6f16-39c8-489e-bc36-7571e2ddc92d None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80601) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 04 21:01:59.770858 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 6/12] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:01:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:01:59.790096 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-c20b6f16-39c8-489e-bc36-7571e2ddc92d None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:01:59.791730 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-c20b6f16-39c8-489e-bc36-7571e2ddc92d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:01:59.793111 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992073de-37f0-4364-b0e0-88ebbdd09783 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:59.816366 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:01:59.890777 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 7/13] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:01:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 21:01:59.893162 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992073de-37f0-4364-b0e0-88ebbdd09783 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:59.912270 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20b6f16-39c8-489e-bc36-7571e2ddc92d None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:59.928143 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74156ac1-202e-4b34-8fe5-ce154932a591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:59.928799 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74156ac1-202e-4b34-8fe5-ce154932a591 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:59.929086 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74156ac1-202e-4b34-8fe5-ce154932a591 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:59.929373 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74156ac1-202e-4b34-8fe5-ce154932a591 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:59.940431 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74156ac1-202e-4b34-8fe5-ce154932a591 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:01:59.949957 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-74156ac1-202e-4b34-8fe5-ce154932a591 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:01:59.949957 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:01:59.949957 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:01:59.949957 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:01:59.949957 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:01:59.949957 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:01:59.949957 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:01:59.949957 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:01:59.949957 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:01:59.949957 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:01:59.949957 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:01:59.949957 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:01:59.949957 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:01:59.949957 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:01:59.949957 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:01:59.949957 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:01:59.950976 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:01:59.950976 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:01:59.950976 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:01:59.950976 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:01:59.950976 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:01:59.950976 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:01:59.950976 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:01:59.950976 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:01:59.950976 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:01:59.950976 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:01:59.950976 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:01:59.950976 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:01:59.950976 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:01:59.950976 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:01:59.950976 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:01:59.950976 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:01:59.952136 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:01:59.952136 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:01:59.952136 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:01:59.952136 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:01:59.952136 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:01:59.952136 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:01:59.952136 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:01:59.952136 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:01:59.952136 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:01:59.952136 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:01:59.952136 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:01:59.952136 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:01:59.952136 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 7/14] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Aug 4 21:01:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:01:59.957224 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74156ac1-202e-4b34-8fe5-ce154932a591 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:00.017102 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20b6f16-39c8-489e-bc36-7571e2ddc92d None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:00.019363 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:00.044357 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:00.055157 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a085d79-6b02-4b51-928f-fb3987faa08a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:00.056070 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a085d79-6b02-4b51-928f-fb3987faa08a None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:00.056358 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a085d79-6b02-4b51-928f-fb3987faa08a None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:00.056634 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a085d79-6b02-4b51-928f-fb3987faa08a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:00.061715 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74156ac1-202e-4b34-8fe5-ce154932a591 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:00.110606 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1cb013-91fb-431f-a9cd-4493ffb769d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:00.111224 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1cb013-91fb-431f-a9cd-4493ffb769d3 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:00.111541 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1cb013-91fb-431f-a9cd-4493ffb769d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:00.111815 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1cb013-91fb-431f-a9cd-4493ffb769d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:00.112555 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1cb013-91fb-431f-a9cd-4493ffb769d3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:00.112850 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1cb013-91fb-431f-a9cd-4493ffb769d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:00.114258 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1cb013-91fb-431f-a9cd-4493ffb769d3 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:00.127860 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 8/15] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Aug 4 21:01:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:02:00.128261 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a085d79-6b02-4b51-928f-fb3987faa08a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:00.129807 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a085d79-6b02-4b51-928f-fb3987faa08a None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:00.133481 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1cb013-91fb-431f-a9cd-4493ffb769d3 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:00.233534 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 8/16] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Aug 4 21:01:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 04 21:02:00.240167 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a085d79-6b02-4b51-928f-fb3987faa08a None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:00.243341 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:00.336658 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1cb013-91fb-431f-a9cd-4493ffb769d3 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:00.342864 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a085d79-6b02-4b51-928f-fb3987faa08a None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:00.370388 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4746e33-daae-4dcf-b3e3-51bc23e70dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:00.371035 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4746e33-daae-4dcf-b3e3-51bc23e70dbb None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:00.371348 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4746e33-daae-4dcf-b3e3-51bc23e70dbb None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:00.371621 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4746e33-daae-4dcf-b3e3-51bc23e70dbb None admin] PATH_INFO: `/v3/users/bc6bad33dfb24427b1ccf30de722b45a` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:00.372779 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4746e33-daae-4dcf-b3e3-51bc23e70dbb None admin] RBAC: Authorizing `identity:get_user(user_id=bc6bad33dfb24427b1ccf30de722b45a)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:00.374315 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4746e33-daae-4dcf-b3e3-51bc23e70dbb None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:00.375575 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eb574f-c3b4-498f-a853-07092418cc11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:00.375979 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 9/17] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Aug 4 21:02:00 2022] GET /identity/v3/users/bc6bad33dfb24427b1ccf30de722b45a => generated 247 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:02:00.376441 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb574f-c3b4-498f-a853-07092418cc11 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:00.376727 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb574f-c3b4-498f-a853-07092418cc11 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:00.377005 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb574f-c3b4-498f-a853-07092418cc11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:00.381413 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4746e33-daae-4dcf-b3e3-51bc23e70dbb None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:00.395849 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19eb574f-c3b4-498f-a853-07092418cc11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:00.396296 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:00.397312 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19eb574f-c3b4-498f-a853-07092418cc11 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:00.416097 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-19eb574f-c3b4-498f-a853-07092418cc11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:02:00.416097 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:02:00.416097 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:02:00.416097 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:02:00.416097 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:02:00.416097 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:02:00.416097 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:02:00.416097 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:02:00.416097 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:02:00.416097 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:02:00.416097 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:02:00.416097 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:02:00.416097 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:02:00.416097 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:02:00.416097 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:02:00.416979 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:02:00.416979 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:02:00.416979 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:02:00.416979 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:02:00.416979 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:02:00.416979 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:02:00.416979 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:02:00.416979 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:02:00.416979 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:02:00.416979 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:02:00.416979 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:02:00.416979 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:02:00.416979 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:02:00.416979 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:02:00.416979 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:02:00.416979 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:02:00.416979 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:02:00.417786 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:02:00.417786 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:02:00.417786 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:02:00.417786 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:02:00.417786 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 9/18] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:02:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:02:00.423136 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eb574f-c3b4-498f-a853-07092418cc11 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:00.479245 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:00.483650 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4746e33-daae-4dcf-b3e3-51bc23e70dbb None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:00.518678 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac73d09-6523-4f9b-8f81-4af7c36f93a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:00.519377 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac73d09-6523-4f9b-8f81-4af7c36f93a9 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:00.520339 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac73d09-6523-4f9b-8f81-4af7c36f93a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:00.520646 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac73d09-6523-4f9b-8f81-4af7c36f93a9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:00.522038 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac73d09-6523-4f9b-8f81-4af7c36f93a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:00.523739 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac73d09-6523-4f9b-8f81-4af7c36f93a9 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:00.523861 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eb574f-c3b4-498f-a853-07092418cc11 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:00.526078 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 10/19] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:02:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:02:00.531573 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac73d09-6523-4f9b-8f81-4af7c36f93a9 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:00.559742 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658137d7-5b73-43c8-bc6a-e5c3ba709277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:00.560406 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658137d7-5b73-43c8-bc6a-e5c3ba709277 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:00.560689 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658137d7-5b73-43c8-bc6a-e5c3ba709277 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:00.560972 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658137d7-5b73-43c8-bc6a-e5c3ba709277 None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:00.561670 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658137d7-5b73-43c8-bc6a-e5c3ba709277 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:00.561979 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658137d7-5b73-43c8-bc6a-e5c3ba709277 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:00.563408 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658137d7-5b73-43c8-bc6a-e5c3ba709277 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:00.580706 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 10/20] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:02:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:02:00.614194 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:00.634026 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac73d09-6523-4f9b-8f81-4af7c36f93a9 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:00.670831 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791b0e66-42fb-4e50-a808-6d93bb933014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:00.671508 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791b0e66-42fb-4e50-a808-6d93bb933014 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:00.671770 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791b0e66-42fb-4e50-a808-6d93bb933014 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:00.672045 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791b0e66-42fb-4e50-a808-6d93bb933014 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:00.672853 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791b0e66-42fb-4e50-a808-6d93bb933014 None admin] RBAC: Adding query filter params (role.id=a432dd4735944ecfab2fa5ce1705d356, scope.domain.id=default, user.id=bc6bad33dfb24427b1ccf30de722b45a) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:00.673163 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791b0e66-42fb-4e50-a808-6d93bb933014 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:00.674659 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791b0e66-42fb-4e50-a808-6d93bb933014 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:00.682336 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 11/21] 10.222.0.25 () {56 vars in 1322 bytes} [Thu Aug 4 21:02:00 2022] GET /identity/v3/role_assignments?user.id=bc6bad33dfb24427b1ccf30de722b45a&scope.domain.id=default&role.id=a432dd4735944ecfab2fa5ce1705d356 => generated 233 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:02:03.154706 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca764c7-1ada-491e-af9e-ea9a7497a851 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:03.155213 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca764c7-1ada-491e-af9e-ea9a7497a851 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:03.155285 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca764c7-1ada-491e-af9e-ea9a7497a851 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:03.156650 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 11/22] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:02:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:02:03.162427 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097213e1-7f34-44e1-a88b-2b0f28ad943f None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:03.162825 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097213e1-7f34-44e1-a88b-2b0f28ad943f None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:03.163020 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097213e1-7f34-44e1-a88b-2b0f28ad943f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:03.192766 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-097213e1-7f34-44e1-a88b-2b0f28ad943f None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:03.194007 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-097213e1-7f34-44e1-a88b-2b0f28ad943f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:03.289998 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 12/23] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:02:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:02:03.318855 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f028273a-2568-41a8-9141-1ef9f863df3d None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:03.319135 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f028273a-2568-41a8-9141-1ef9f863df3d None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:03.319482 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f028273a-2568-41a8-9141-1ef9f863df3d None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:03.321096 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 12/24] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:02:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:02:03.328689 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f028273a-2568-41a8-9141-1ef9f863df3d None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:03.329236 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23de91b8-bc72-4061-90eb-73b248c6009b None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:03.329582 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23de91b8-bc72-4061-90eb-73b248c6009b None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:03.329812 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23de91b8-bc72-4061-90eb-73b248c6009b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:03.356594 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-23de91b8-bc72-4061-90eb-73b248c6009b None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:03.357704 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-23de91b8-bc72-4061-90eb-73b248c6009b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:03.451461 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 13/25] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:02:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:02:03.452792 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63fed63-dc86-411e-97c5-74cbd55774b5 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:03.453087 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63fed63-dc86-411e-97c5-74cbd55774b5 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:03.453368 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63fed63-dc86-411e-97c5-74cbd55774b5 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:03.454936 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 14/26] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:02:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:02:03.463235 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c57525a-0edd-4b59-8bf6-b3ffabdf658e None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:03.463549 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c57525a-0edd-4b59-8bf6-b3ffabdf658e None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:03.463834 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c57525a-0edd-4b59-8bf6-b3ffabdf658e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:03.474659 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:03.490787 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-8c57525a-0edd-4b59-8bf6-b3ffabdf658e None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:03.492007 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-8c57525a-0edd-4b59-8bf6-b3ffabdf658e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:03.525168 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f028273a-2568-41a8-9141-1ef9f863df3d None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:03.556127 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:03.565867 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 15/27] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:02:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 21:02:03.567434 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c57525a-0edd-4b59-8bf6-b3ffabdf658e None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:03.599209 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e88f675-c260-4f4c-9e17-73e88fa891c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:03.599972 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e88f675-c260-4f4c-9e17-73e88fa891c6 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:03.600261 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e88f675-c260-4f4c-9e17-73e88fa891c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:03.600544 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e88f675-c260-4f4c-9e17-73e88fa891c6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:03.633961 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e88f675-c260-4f4c-9e17-73e88fa891c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:02:03.642524 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-1e88f675-c260-4f4c-9e17-73e88fa891c6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:02:03.642524 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:02:03.642524 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:02:03.642524 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:02:03.642524 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:02:03.642524 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:02:03.642524 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:02:03.642524 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:02:03.642524 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:02:03.642524 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:02:03.642524 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:02:03.642524 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:02:03.642524 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:02:03.642524 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:02:03.642524 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:02:03.642524 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:02:03.643426 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:02:03.643426 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:02:03.643426 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:02:03.643426 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:02:03.643426 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:02:03.643426 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:02:03.643426 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:02:03.643426 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:02:03.643426 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:02:03.643426 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:02:03.643426 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:02:03.643426 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:02:03.643426 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:02:03.643426 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:02:03.643426 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:02:03.643426 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:02:03.644199 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:02:03.644199 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:02:03.644199 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:02:03.644199 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:02:03.644199 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:02:03.644199 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:02:03.644199 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:02:03.644199 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:02:03.644199 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:02:03.644199 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:02:03.644199 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:02:03.644199 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:02:03.651268 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 13/28] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Aug 4 21:02:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:02:03.652100 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e88f675-c260-4f4c-9e17-73e88fa891c6 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:03.669350 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c57525a-0edd-4b59-8bf6-b3ffabdf658e None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:03.708202 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5f3388-3bed-4d7a-8014-bfb77f2a5c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:03.708768 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5f3388-3bed-4d7a-8014-bfb77f2a5c9d None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:03.709052 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5f3388-3bed-4d7a-8014-bfb77f2a5c9d None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:03.709331 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5f3388-3bed-4d7a-8014-bfb77f2a5c9d None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:03.710072 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5f3388-3bed-4d7a-8014-bfb77f2a5c9d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:03.711539 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5f3388-3bed-4d7a-8014-bfb77f2a5c9d None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:03.715640 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:03.884068 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e88f675-c260-4f4c-9e17-73e88fa891c6 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:03.920231 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731646cd-1f1a-4a5d-a4e5-73690d672803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:03.920860 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731646cd-1f1a-4a5d-a4e5-73690d672803 None admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:03.921145 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731646cd-1f1a-4a5d-a4e5-73690d672803 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:03.921479 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731646cd-1f1a-4a5d-a4e5-73690d672803 None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:03.922201 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731646cd-1f1a-4a5d-a4e5-73690d672803 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:03.923683 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731646cd-1f1a-4a5d-a4e5-73690d672803 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:03.935354 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 16/29] 10.222.0.25 () {60 vars in 1131 bytes} [Thu Aug 4 21:02:03 2022] POST /identity/v3/roles => generated 219 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:02:03.936351 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5f3388-3bed-4d7a-8014-bfb77f2a5c9d None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:04.070730 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5f3388-3bed-4d7a-8014-bfb77f2a5c9d None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:04.120754 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a8230c-1594-4da5-a2c4-f604c5b58959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:04.121418 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a8230c-1594-4da5-a2c4-f604c5b58959 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:04.121710 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a8230c-1594-4da5-a2c4-f604c5b58959 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:04.121990 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a8230c-1594-4da5-a2c4-f604c5b58959 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:04.122827 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a8230c-1594-4da5-a2c4-f604c5b58959 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:04.123147 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a8230c-1594-4da5-a2c4-f604c5b58959 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:04.124812 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a8230c-1594-4da5-a2c4-f604c5b58959 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:04.138227 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 17/30] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Aug 4 21:02:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:04.139707 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f3bca3-7a82-49f4-b6bc-c9f68ab4a1a3 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:04.140011 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f3bca3-7a82-49f4-b6bc-c9f68ab4a1a3 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:04.141225 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f3bca3-7a82-49f4-b6bc-c9f68ab4a1a3 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:04.141672 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 18/31] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:02:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:02:04.147342 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca92ba98-6a60-46cc-8020-5815f0b39d65 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:04.147342 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca92ba98-6a60-46cc-8020-5815f0b39d65 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:04.147342 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca92ba98-6a60-46cc-8020-5815f0b39d65 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:04.147342 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 19/32] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:02:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:02:04.151098 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca92ba98-6a60-46cc-8020-5815f0b39d65 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:04.172387 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 14/33] 10.222.0.25 () {60 vars in 1131 bytes} [Thu Aug 4 21:02:03 2022] POST /identity/v3/roles => generated 225 bytes in 521 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:02:04.173871 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09a9414-7690-41b5-bf9b-baab62d6bf55 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:04.174217 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09a9414-7690-41b5-bf9b-baab62d6bf55 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:04.174503 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09a9414-7690-41b5-bf9b-baab62d6bf55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:04.201513 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-e09a9414-7690-41b5-bf9b-baab62d6bf55 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:04.203590 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-e09a9414-7690-41b5-bf9b-baab62d6bf55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:04.237770 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:04.260834 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:04.307055 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca92ba98-6a60-46cc-8020-5815f0b39d65 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:04.327440 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 15/34] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:02:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:02:04.327440 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cb3418-fcca-475b-94f5-f840023b86a0 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:04.327440 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cb3418-fcca-475b-94f5-f840023b86a0 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:04.328316 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cb3418-fcca-475b-94f5-f840023b86a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:04.351545 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7331aa-8600-44e4-b049-f805533218a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:04.360081 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7331aa-8600-44e4-b049-f805533218a9 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:04.360081 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7331aa-8600-44e4-b049-f805533218a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:04.360081 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7331aa-8600-44e4-b049-f805533218a9 None admin] PATH_INFO: `/v3/users/bc6bad33dfb24427b1ccf30de722b45a` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:04.360081 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7331aa-8600-44e4-b049-f805533218a9 None admin] RBAC: Authorizing `identity:get_user(user_id=bc6bad33dfb24427b1ccf30de722b45a)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:04.360081 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7331aa-8600-44e4-b049-f805533218a9 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:04.360081 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 20/35] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Aug 4 21:02:04 2022] GET /identity/v3/users/bc6bad33dfb24427b1ccf30de722b45a => generated 247 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:04.360530 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-d5cb3418-fcca-475b-94f5-f840023b86a0 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:04.360530 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-d5cb3418-fcca-475b-94f5-f840023b86a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:04.364053 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7331aa-8600-44e4-b049-f805533218a9 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:04.452097 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 16/36] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:02:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:02:04.453408 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cb3418-fcca-475b-94f5-f840023b86a0 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:04.494665 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7331aa-8600-44e4-b049-f805533218a9 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:04.522056 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:04.532394 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5bd93c-0100-4023-971a-83faf33d312a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:04.533312 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5bd93c-0100-4023-971a-83faf33d312a None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:04.533609 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5bd93c-0100-4023-971a-83faf33d312a None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:04.533890 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5bd93c-0100-4023-971a-83faf33d312a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:04.535684 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5bd93c-0100-4023-971a-83faf33d312a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:04.537232 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5bd93c-0100-4023-971a-83faf33d312a None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:04.538896 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 21/37] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:02:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:04.539682 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5bd93c-0100-4023-971a-83faf33d312a None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:04.554882 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:04.560569 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cb3418-fcca-475b-94f5-f840023b86a0 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:04.593527 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3890269-1f96-4ce8-a4a1-cf6aefef7f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:04.594109 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3890269-1f96-4ce8-a4a1-cf6aefef7f0c None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:04.594399 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3890269-1f96-4ce8-a4a1-cf6aefef7f0c None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:04.594680 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3890269-1f96-4ce8-a4a1-cf6aefef7f0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:04.605275 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3890269-1f96-4ce8-a4a1-cf6aefef7f0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:04.606881 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3890269-1f96-4ce8-a4a1-cf6aefef7f0c None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:04.622677 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-a3890269-1f96-4ce8-a4a1-cf6aefef7f0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:02:04.622677 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:02:04.622677 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:02:04.622677 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:02:04.622677 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:02:04.622677 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:02:04.622677 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:02:04.622677 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:02:04.622677 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:02:04.622677 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:02:04.622677 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:02:04.622677 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:02:04.622677 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:02:04.622677 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:02:04.622677 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:02:04.623646 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:02:04.623646 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:02:04.623646 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:02:04.623646 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:02:04.623646 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:02:04.623646 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:02:04.623646 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:02:04.623646 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:02:04.623646 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:02:04.623646 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:02:04.623646 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:02:04.623646 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:02:04.623646 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:02:04.623646 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:02:04.623646 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:02:04.623646 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:02:04.623646 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:02:04.624452 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:02:04.624452 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:02:04.624452 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:02:04.624452 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:02:04.624452 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 17/38] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:02:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:02:04.625185 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c71faea-bad8-4fd6-a9c3-92bd48d88a09 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:04.625485 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c71faea-bad8-4fd6-a9c3-92bd48d88a09 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:04.625766 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c71faea-bad8-4fd6-a9c3-92bd48d88a09 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:04.627076 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 18/39] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:02:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:02:04.627801 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c71faea-bad8-4fd6-a9c3-92bd48d88a09 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:04.643408 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5bd93c-0100-4023-971a-83faf33d312a None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:04.678510 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c100570-11dc-4512-aa1f-4131f91c460f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:04.679165 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c100570-11dc-4512-aa1f-4131f91c460f None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:04.679489 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c100570-11dc-4512-aa1f-4131f91c460f None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:04.679776 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c100570-11dc-4512-aa1f-4131f91c460f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:04.680952 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c100570-11dc-4512-aa1f-4131f91c460f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:04.682437 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c100570-11dc-4512-aa1f-4131f91c460f None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:04.684090 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 22/40] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:02:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:04.684845 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c100570-11dc-4512-aa1f-4131f91c460f None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:04.734442 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c71faea-bad8-4fd6-a9c3-92bd48d88a09 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:04.744468 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:04.773635 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c312d5a-7979-4752-959f-6e1ac40a8ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:04.774325 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c312d5a-7979-4752-959f-6e1ac40a8ebd None admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:04.774623 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c312d5a-7979-4752-959f-6e1ac40a8ebd None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:04.774908 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c312d5a-7979-4752-959f-6e1ac40a8ebd None admin] PATH_INFO: `/v3/domains/default/users/bc6bad33dfb24427b1ccf30de722b45a/roles/a432dd4735944ecfab2fa5ce1705d356` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:04.777408 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c312d5a-7979-4752-959f-6e1ac40a8ebd None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=bc6bad33dfb24427b1ccf30de722b45a, role_id=a432dd4735944ecfab2fa5ce1705d356)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:04.778883 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c312d5a-7979-4752-959f-6e1ac40a8ebd None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:04.790835 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c100570-11dc-4512-aa1f-4131f91c460f None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:04.824047 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106c1742-fe92-4995-aedf-66d65d13bb36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:04.824795 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106c1742-fe92-4995-aedf-66d65d13bb36 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:04.825089 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106c1742-fe92-4995-aedf-66d65d13bb36 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:04.825371 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106c1742-fe92-4995-aedf-66d65d13bb36 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:04.826072 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106c1742-fe92-4995-aedf-66d65d13bb36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:04.826391 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106c1742-fe92-4995-aedf-66d65d13bb36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:04.827839 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106c1742-fe92-4995-aedf-66d65d13bb36 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:04.845528 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 23/41] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:02:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:04.846661 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca11845-0f2d-4958-8e60-62731255f8c7 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:04.846959 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca11845-0f2d-4958-8e60-62731255f8c7 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:04.847242 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca11845-0f2d-4958-8e60-62731255f8c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:04.865120 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-4ca11845-0f2d-4958-8e60-62731255f8c7 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:04.866647 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-4ca11845-0f2d-4958-8e60-62731255f8c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:04.881504 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 19/42] 10.222.0.25 () {58 vars in 1361 bytes} [Thu Aug 4 21:02:04 2022] PUT /identity/v3/domains/default/users/bc6bad33dfb24427b1ccf30de722b45a/roles/a432dd4735944ecfab2fa5ce1705d356 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:02:04.882316 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c312d5a-7979-4752-959f-6e1ac40a8ebd None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:04.921813 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:04.948937 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 24/43] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:02:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:02:04.957607 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca11845-0f2d-4958-8e60-62731255f8c7 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:05.007267 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:05.049061 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c312d5a-7979-4752-959f-6e1ac40a8ebd None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:05.065943 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca11845-0f2d-4958-8e60-62731255f8c7 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:05.085693 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c785921-96b4-44c0-9ac5-2a8c38702400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:05.086326 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c785921-96b4-44c0-9ac5-2a8c38702400 None admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:05.086608 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c785921-96b4-44c0-9ac5-2a8c38702400 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:05.086882 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c785921-96b4-44c0-9ac5-2a8c38702400 None admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:05.087629 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c785921-96b4-44c0-9ac5-2a8c38702400 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:05.089077 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c785921-96b4-44c0-9ac5-2a8c38702400 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:05.102616 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abee830-df38-45a5-940e-33d45f43fc0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:05.103170 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abee830-df38-45a5-940e-33d45f43fc0f None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:05.103532 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abee830-df38-45a5-940e-33d45f43fc0f None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:05.103809 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abee830-df38-45a5-940e-33d45f43fc0f None admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:05.104601 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abee830-df38-45a5-940e-33d45f43fc0f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:05.105954 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abee830-df38-45a5-940e-33d45f43fc0f None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:05.202399 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 20/44] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Aug 4 21:02:04 2022] POST /identity/v3/projects => generated 312 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:02:05.203209 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c785921-96b4-44c0-9ac5-2a8c38702400 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:05.234213 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 25/45] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Aug 4 21:02:04 2022] POST /identity/v3/projects => generated 301 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:02:05.296510 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:05.349783 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c785921-96b4-44c0-9ac5-2a8c38702400 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:05.392640 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93074f0-3e42-4c27-b612-073663f5b49e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:05.393616 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93074f0-3e42-4c27-b612-073663f5b49e None admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:05.394082 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93074f0-3e42-4c27-b612-073663f5b49e None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:05.394535 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93074f0-3e42-4c27-b612-073663f5b49e None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:05.395462 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93074f0-3e42-4c27-b612-073663f5b49e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:05.397019 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93074f0-3e42-4c27-b612-073663f5b49e None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:05.471437 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 21/46] 10.222.0.25 () {60 vars in 1131 bytes} [Thu Aug 4 21:02:05 2022] POST /identity/v3/roles => generated 223 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:02:06.922999 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2a9dc0-dcab-4b3d-9eb2-aa3287fe2c80 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:06.923519 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2a9dc0-dcab-4b3d-9eb2-aa3287fe2c80 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:06.923600 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2a9dc0-dcab-4b3d-9eb2-aa3287fe2c80 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:06.924952 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 26/47] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:02:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:02:06.930238 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355b157b-8f6d-424a-93d0-5fbc897cc7d9 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:06.930663 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355b157b-8f6d-424a-93d0-5fbc897cc7d9 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:06.930928 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355b157b-8f6d-424a-93d0-5fbc897cc7d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:06.949432 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-355b157b-8f6d-424a-93d0-5fbc897cc7d9 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:06.950623 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-355b157b-8f6d-424a-93d0-5fbc897cc7d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:07.015910 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 22/48] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:02:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:02:07.039341 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2a9dc0-dcab-4b3d-9eb2-aa3287fe2c80 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:07.138036 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2a9dc0-dcab-4b3d-9eb2-aa3287fe2c80 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:07.139586 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:07.171286 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc668b9-4b3d-4438-a3f2-8f059b4644f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:07.171970 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc668b9-4b3d-4438-a3f2-8f059b4644f9 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:07.172422 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc668b9-4b3d-4438-a3f2-8f059b4644f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:07.173189 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc668b9-4b3d-4438-a3f2-8f059b4644f9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:07.183529 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc668b9-4b3d-4438-a3f2-8f059b4644f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:02:07.188462 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-8fc668b9-4b3d-4438-a3f2-8f059b4644f9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:02:07.188462 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:02:07.188462 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:02:07.188462 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:02:07.188462 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:02:07.188462 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:02:07.188462 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:02:07.188462 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:02:07.188462 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:02:07.188462 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:02:07.188462 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:02:07.188462 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:02:07.188462 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:02:07.188462 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:02:07.188462 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:02:07.188462 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:02:07.189284 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:02:07.189284 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:02:07.189284 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:02:07.189284 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:02:07.189284 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:02:07.189284 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:02:07.189284 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:02:07.189284 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:02:07.189284 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:02:07.189284 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:02:07.189284 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:02:07.189284 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:02:07.189284 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:02:07.189284 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:02:07.189284 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:02:07.189284 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:02:07.190022 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:02:07.190022 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:02:07.190022 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:02:07.190022 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:02:07.190022 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:02:07.190022 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:02:07.190022 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:02:07.190022 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:02:07.190022 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:02:07.190022 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:02:07.190022 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:02:07.190022 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:02:07.190022 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 27/49] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Aug 4 21:02:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:02:07.196098 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355b157b-8f6d-424a-93d0-5fbc897cc7d9 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:07.248378 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:07.263390 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4766fc63-396c-49cb-808e-839009eb2451 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:07.263870 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4766fc63-396c-49cb-808e-839009eb2451 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:07.264155 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4766fc63-396c-49cb-808e-839009eb2451 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:07.265524 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 28/50] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:02:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 21:02:07.270448 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a5b884-48cc-4fa4-ad8a-0c7ca347981e None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:07.270732 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a5b884-48cc-4fa4-ad8a-0c7ca347981e None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:07.271008 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a5b884-48cc-4fa4-ad8a-0c7ca347981e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:07.289410 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-f5a5b884-48cc-4fa4-ad8a-0c7ca347981e None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:07.290610 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-f5a5b884-48cc-4fa4-ad8a-0c7ca347981e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:07.298672 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355b157b-8f6d-424a-93d0-5fbc897cc7d9 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:07.335728 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b79b566-0b5f-426c-8879-feb166428365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:07.336347 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b79b566-0b5f-426c-8879-feb166428365 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:07.336691 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b79b566-0b5f-426c-8879-feb166428365 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:07.336976 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b79b566-0b5f-426c-8879-feb166428365 None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:07.337687 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b79b566-0b5f-426c-8879-feb166428365 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:07.337989 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b79b566-0b5f-426c-8879-feb166428365 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:07.339384 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b79b566-0b5f-426c-8879-feb166428365 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:07.350677 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 23/51] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Aug 4 21:02:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:07.356083 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 29/52] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:02:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 21:02:07.356297 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b79b566-0b5f-426c-8879-feb166428365 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:07.378524 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a5b884-48cc-4fa4-ad8a-0c7ca347981e None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:07.453164 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b79b566-0b5f-426c-8879-feb166428365 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:07.466324 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:07.466922 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:07.475462 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a5b884-48cc-4fa4-ad8a-0c7ca347981e None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:07.486534 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66854d90-876c-4201-a6e7-8db68b9673db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:07.487181 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66854d90-876c-4201-a6e7-8db68b9673db None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:07.497599 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66854d90-876c-4201-a6e7-8db68b9673db None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:07.497599 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66854d90-876c-4201-a6e7-8db68b9673db None admin] PATH_INFO: `/v3/users/bc6bad33dfb24427b1ccf30de722b45a` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:07.497599 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66854d90-876c-4201-a6e7-8db68b9673db None admin] RBAC: Authorizing `identity:get_user(user_id=bc6bad33dfb24427b1ccf30de722b45a)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:07.497599 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66854d90-876c-4201-a6e7-8db68b9673db None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:07.497599 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 24/53] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Aug 4 21:02:07 2022] GET /identity/v3/users/bc6bad33dfb24427b1ccf30de722b45a => generated 247 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:02:07.497980 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66854d90-876c-4201-a6e7-8db68b9673db None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:07.508971 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaf79a6-6d28-4ab2-bff3-778a5ca5220e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:07.509618 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf79a6-6d28-4ab2-bff3-778a5ca5220e None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:07.509909 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf79a6-6d28-4ab2-bff3-778a5ca5220e None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:07.510192 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf79a6-6d28-4ab2-bff3-778a5ca5220e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:07.511367 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaf79a6-6d28-4ab2-bff3-778a5ca5220e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:07.512829 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaf79a6-6d28-4ab2-bff3-778a5ca5220e None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:07.514421 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 30/54] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:02:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:02:07.519856 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaf79a6-6d28-4ab2-bff3-778a5ca5220e None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:07.599820 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66854d90-876c-4201-a6e7-8db68b9673db None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:07.620882 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaf79a6-6d28-4ab2-bff3-778a5ca5220e None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:07.633254 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbfb0df-9bfd-4811-9585-17b6570393c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:07.633905 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbfb0df-9bfd-4811-9585-17b6570393c2 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:07.634192 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbfb0df-9bfd-4811-9585-17b6570393c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:07.634469 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbfb0df-9bfd-4811-9585-17b6570393c2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:07.635652 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbfb0df-9bfd-4811-9585-17b6570393c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:07.637126 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbfb0df-9bfd-4811-9585-17b6570393c2 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:07.638738 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 25/55] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:02:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:02:07.644507 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbfb0df-9bfd-4811-9585-17b6570393c2 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:07.654069 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f53617-23f7-497c-b6c3-c55d59a5fd09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:07.654703 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f53617-23f7-497c-b6c3-c55d59a5fd09 None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:07.655157 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f53617-23f7-497c-b6c3-c55d59a5fd09 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:07.655472 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f53617-23f7-497c-b6c3-c55d59a5fd09 None admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:07.656162 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f53617-23f7-497c-b6c3-c55d59a5fd09 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:07.657699 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f53617-23f7-497c-b6c3-c55d59a5fd09 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:07.692401 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:07.742148 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbfb0df-9bfd-4811-9585-17b6570393c2 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:07.749872 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 31/56] 10.222.0.25 () {60 vars in 1140 bytes} [Thu Aug 4 21:02:07 2022] POST /identity/v3/projects => generated 298 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:02:07.778787 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b7ee4c-1474-41d9-9460-b66ca4829a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:07.779460 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b7ee4c-1474-41d9-9460-b66ca4829a15 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:07.779749 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b7ee4c-1474-41d9-9460-b66ca4829a15 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:07.780026 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b7ee4c-1474-41d9-9460-b66ca4829a15 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:07.780828 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b7ee4c-1474-41d9-9460-b66ca4829a15 None admin] RBAC: Adding query filter params (role.id=a432dd4735944ecfab2fa5ce1705d356, scope.domain.id=default, user.id=bc6bad33dfb24427b1ccf30de722b45a) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:07.781186 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b7ee4c-1474-41d9-9460-b66ca4829a15 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:07.783166 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b7ee4c-1474-41d9-9460-b66ca4829a15 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:07.792350 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 26/57] 10.222.0.25 () {56 vars in 1322 bytes} [Thu Aug 4 21:02:07 2022] GET /identity/v3/role_assignments?user.id=bc6bad33dfb24427b1ccf30de722b45a&scope.domain.id=default&role.id=a432dd4735944ecfab2fa5ce1705d356 => generated 530 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:02:09.786555 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6364608d-15d9-4df6-9d49-85107ca7bdab None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:09.787066 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6364608d-15d9-4df6-9d49-85107ca7bdab None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:09.787148 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6364608d-15d9-4df6-9d49-85107ca7bdab None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:09.788518 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 32/58] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:02:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:02:09.793666 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ff7190-c902-4211-8a37-16d02403553d None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:09.794067 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ff7190-c902-4211-8a37-16d02403553d None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:09.794231 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ff7190-c902-4211-8a37-16d02403553d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:09.812021 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-65ff7190-c902-4211-8a37-16d02403553d None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:09.813206 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-65ff7190-c902-4211-8a37-16d02403553d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:09.879178 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 27/59] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:02:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:02:09.902671 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6364608d-15d9-4df6-9d49-85107ca7bdab None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:09.918399 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:10.006099 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6364608d-15d9-4df6-9d49-85107ca7bdab None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:10.039558 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc2d609-0d1d-42c9-b291-30ea75e58ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:10.040317 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc2d609-0d1d-42c9-b291-30ea75e58ddd None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:10.040762 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc2d609-0d1d-42c9-b291-30ea75e58ddd None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:10.041679 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc2d609-0d1d-42c9-b291-30ea75e58ddd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:10.042829 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc2d609-0d1d-42c9-b291-30ea75e58ddd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:10.044591 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc2d609-0d1d-42c9-b291-30ea75e58ddd None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:10.045996 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 33/60] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:02:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:10.052151 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ff7190-c902-4211-8a37-16d02403553d None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:10.093161 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:10.148043 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ff7190-c902-4211-8a37-16d02403553d None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:10.180888 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d458ad25-291f-42b4-8424-f4376c2e62ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:10.181502 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d458ad25-291f-42b4-8424-f4376c2e62ba None admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:10.181810 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d458ad25-291f-42b4-8424-f4376c2e62ba None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:10.182086 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d458ad25-291f-42b4-8424-f4376c2e62ba None admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:10.182778 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d458ad25-291f-42b4-8424-f4376c2e62ba None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:10.184252 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d458ad25-291f-42b4-8424-f4376c2e62ba None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:10.292232 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 28/61] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Aug 4 21:02:10 2022] POST /identity/v3/users => generated 275 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:02:13.421088 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5324c7-369e-4387-a611-eeacfffd8225 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:13.421954 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5324c7-369e-4387-a611-eeacfffd8225 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:13.422442 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5324c7-369e-4387-a611-eeacfffd8225 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:13.424315 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 34/62] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:02:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:02:13.429886 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5162c57d-bd13-477f-9391-cc11c16e7cc0 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:13.430638 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5162c57d-bd13-477f-9391-cc11c16e7cc0 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:13.431076 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5162c57d-bd13-477f-9391-cc11c16e7cc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:13.457237 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-5162c57d-bd13-477f-9391-cc11c16e7cc0 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:13.458778 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-5162c57d-bd13-477f-9391-cc11c16e7cc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:13.546000 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 29/63] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:02:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 21:02:13.569982 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5324c7-369e-4387-a611-eeacfffd8225 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:13.655923 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:13.682805 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5324c7-369e-4387-a611-eeacfffd8225 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:13.719941 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5993f69c-a7bc-4b0a-a41b-2e236ccdedea None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:13.720717 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5993f69c-a7bc-4b0a-a41b-2e236ccdedea None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:13.721203 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5993f69c-a7bc-4b0a-a41b-2e236ccdedea None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:13.722829 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 30/64] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:02:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:02:13.729003 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd534d-8fd4-42e3-a6ed-f2be9363cc28 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:13.729635 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd534d-8fd4-42e3-a6ed-f2be9363cc28 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:13.730188 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd534d-8fd4-42e3-a6ed-f2be9363cc28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:13.731854 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e0347d-ade9-4d72-b320-6f67946c64bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:13.732515 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e0347d-ade9-4d72-b320-6f67946c64bd None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:13.732799 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e0347d-ade9-4d72-b320-6f67946c64bd None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:13.733075 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e0347d-ade9-4d72-b320-6f67946c64bd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:13.751437 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e0347d-ade9-4d72-b320-6f67946c64bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:02:13.756396 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-f1fd534d-8fd4-42e3-a6ed-f2be9363cc28 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:13.757668 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-82e0347d-ade9-4d72-b320-6f67946c64bd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 21:02:13.757668 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:02:13.757668 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:02:13.757668 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:02:13.757668 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:02:13.757668 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:02:13.757668 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:02:13.757668 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:02:13.757668 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:02:13.757668 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:02:13.757668 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:02:13.757668 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:02:13.757668 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:02:13.757668 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:02:13.757668 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:02:13.757668 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:02:13.758500 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:02:13.758500 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:02:13.758500 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:02:13.758500 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:02:13.758500 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:02:13.758500 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:02:13.758500 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:02:13.758500 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:02:13.758500 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:02:13.758500 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:02:13.758500 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:02:13.758500 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:02:13.758500 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:02:13.758500 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:02:13.758500 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:02:13.758500 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:02:13.759244 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:02:13.759244 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:02:13.759244 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:02:13.759244 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:02:13.759244 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:02:13.759244 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:02:13.759244 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:02:13.759244 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:02:13.759244 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:02:13.759244 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:02:13.759244 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 21:02:13.759244 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:02:13.759244 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 35/65] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Aug 4 21:02:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:02:13.760835 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-f1fd534d-8fd4-42e3-a6ed-f2be9363cc28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:13.766604 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e0347d-ade9-4d72-b320-6f67946c64bd None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:13.809996 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:13.884020 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 31/66] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:02:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:02:13.885177 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25864e80-a60c-46ae-b9fa-b5b11dd45544 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:13.885464 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25864e80-a60c-46ae-b9fa-b5b11dd45544 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:13.885746 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25864e80-a60c-46ae-b9fa-b5b11dd45544 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:13.887006 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 32/67] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:02:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:02:13.905898 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655af485-9279-4936-ae74-5fb232394c61 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:13.906390 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655af485-9279-4936-ae74-5fb232394c61 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:13.906842 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655af485-9279-4936-ae74-5fb232394c61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:13.926078 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e0347d-ade9-4d72-b320-6f67946c64bd None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:13.933781 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-655af485-9279-4936-ae74-5fb232394c61 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:13.934972 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-655af485-9279-4936-ae74-5fb232394c61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:13.969337 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef67fae4-51eb-4f95-b644-6d7a7e5020a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:13.970360 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef67fae4-51eb-4f95-b644-6d7a7e5020a3 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:13.971222 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef67fae4-51eb-4f95-b644-6d7a7e5020a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:13.971985 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef67fae4-51eb-4f95-b644-6d7a7e5020a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:13.973907 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef67fae4-51eb-4f95-b644-6d7a7e5020a3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:13.974671 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef67fae4-51eb-4f95-b644-6d7a7e5020a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:13.977202 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef67fae4-51eb-4f95-b644-6d7a7e5020a3 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:13.988696 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 36/68] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Aug 4 21:02:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:13.990197 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef67fae4-51eb-4f95-b644-6d7a7e5020a3 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:13.994211 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:14.013547 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 33/69] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:02:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 21:02:14.014756 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2795c51b-951e-4139-a012-741b875fbdd8 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:14.015300 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2795c51b-951e-4139-a012-741b875fbdd8 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:14.015844 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2795c51b-951e-4139-a012-741b875fbdd8 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:14.017344 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 34/70] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:02:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:02:14.018280 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2795c51b-951e-4139-a012-741b875fbdd8 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:14.110481 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef67fae4-51eb-4f95-b644-6d7a7e5020a3 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:14.119046 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:14.131419 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2795c51b-951e-4139-a012-741b875fbdd8 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:14.143013 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55479ea-aa82-4f69-b177-25969eb4ad1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:14.143684 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55479ea-aa82-4f69-b177-25969eb4ad1b None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:14.143972 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55479ea-aa82-4f69-b177-25969eb4ad1b None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:14.144769 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55479ea-aa82-4f69-b177-25969eb4ad1b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:14.154678 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55479ea-aa82-4f69-b177-25969eb4ad1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:02:14.160919 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-b55479ea-aa82-4f69-b177-25969eb4ad1b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:02:14.160919 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:02:14.160919 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:02:14.160919 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:02:14.160919 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:02:14.160919 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:02:14.160919 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:02:14.160919 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:02:14.160919 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:02:14.160919 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:02:14.160919 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:02:14.160919 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:02:14.160919 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:02:14.160919 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:02:14.160919 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:02:14.160919 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:02:14.161863 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:02:14.161863 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:02:14.161863 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:02:14.161863 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:02:14.161863 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:02:14.161863 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:02:14.161863 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:02:14.161863 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:02:14.161863 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:02:14.161863 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:02:14.161863 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:02:14.161863 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:02:14.161863 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:02:14.161863 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:02:14.161863 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:02:14.161863 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:02:14.162633 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:02:14.162633 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:02:14.162633 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:02:14.162633 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:02:14.162633 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:02:14.162633 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:02:14.162633 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:02:14.162633 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:02:14.162633 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:02:14.162633 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:02:14.162633 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:02:14.162633 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:02:14.162633 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 37/71] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Aug 4 21:02:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:02:14.163256 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e05b22-c3a9-4ea9-a174-e1779e33a5aa None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:14.163474 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e05b22-c3a9-4ea9-a174-e1779e33a5aa None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:14.163798 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e05b22-c3a9-4ea9-a174-e1779e33a5aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:14.169023 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e984a4e-5c72-42b0-9d3b-c6e8ccdac30f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:14.169678 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e984a4e-5c72-42b0-9d3b-c6e8ccdac30f None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:14.169965 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e984a4e-5c72-42b0-9d3b-c6e8ccdac30f None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:14.170244 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e984a4e-5c72-42b0-9d3b-c6e8ccdac30f None admin] PATH_INFO: `/v3/users/d562ab3d5d014f1dbf980dedad3e33c7` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:14.180851 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-85e05b22-c3a9-4ea9-a174-e1779e33a5aa None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:14.182056 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-85e05b22-c3a9-4ea9-a174-e1779e33a5aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:14.185851 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e984a4e-5c72-42b0-9d3b-c6e8ccdac30f None admin] RBAC: Authorizing `identity:get_user(user_id=d562ab3d5d014f1dbf980dedad3e33c7)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:14.187340 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e984a4e-5c72-42b0-9d3b-c6e8ccdac30f None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:14.189044 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 35/72] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Aug 4 21:02:14 2022] GET /identity/v3/users/d562ab3d5d014f1dbf980dedad3e33c7 => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:14.189805 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e984a4e-5c72-42b0-9d3b-c6e8ccdac30f None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:14.243094 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:14.254161 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 38/73] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:02:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:02:14.255428 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af55a386-985e-4e96-9e85-7fbdf5309183 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:14.255737 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af55a386-985e-4e96-9e85-7fbdf5309183 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:14.256067 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af55a386-985e-4e96-9e85-7fbdf5309183 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:14.259561 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 39/74] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:02:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:02:14.259561 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af55a386-985e-4e96-9e85-7fbdf5309183 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:14.300748 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e984a4e-5c72-42b0-9d3b-c6e8ccdac30f None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:14.341689 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dde70c4-1bc8-4e63-acec-e45ce4c42511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:14.342337 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dde70c4-1bc8-4e63-acec-e45ce4c42511 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:14.342623 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dde70c4-1bc8-4e63-acec-e45ce4c42511 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:14.342893 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dde70c4-1bc8-4e63-acec-e45ce4c42511 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:14.352778 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dde70c4-1bc8-4e63-acec-e45ce4c42511 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:02:14.357710 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-5dde70c4-1bc8-4e63-acec-e45ce4c42511 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 21:02:14.357710 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:02:14.357710 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:02:14.357710 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:02:14.357710 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:02:14.357710 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:02:14.357710 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:02:14.357710 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:02:14.357710 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:02:14.357710 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:02:14.357710 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:02:14.357710 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:02:14.357710 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:02:14.357710 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:02:14.357710 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:02:14.357710 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:02:14.358651 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:02:14.358651 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:02:14.358651 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:02:14.358651 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:02:14.358651 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:02:14.358651 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:02:14.358651 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:02:14.358651 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:02:14.358651 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:02:14.358651 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:02:14.358651 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:02:14.358651 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:02:14.358651 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:02:14.358651 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:02:14.358651 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:02:14.358651 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:02:14.359447 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:02:14.359447 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:02:14.359447 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:02:14.359447 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:02:14.359447 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:02:14.359447 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:02:14.359447 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:02:14.359447 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:02:14.359447 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:02:14.359447 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:02:14.359447 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 21:02:14.359447 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:02:14.359447 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 36/75] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Aug 4 21:02:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:02:14.360084 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dde70c4-1bc8-4e63-acec-e45ce4c42511 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:14.371933 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:14.377303 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af55a386-985e-4e96-9e85-7fbdf5309183 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:14.411935 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebd6739-dedb-463c-98a3-1c777ac68897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:14.412559 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebd6739-dedb-463c-98a3-1c777ac68897 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:14.412854 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebd6739-dedb-463c-98a3-1c777ac68897 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:14.413119 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebd6739-dedb-463c-98a3-1c777ac68897 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:14.413837 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebd6739-dedb-463c-98a3-1c777ac68897 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:14.414145 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebd6739-dedb-463c-98a3-1c777ac68897 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:14.415603 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebd6739-dedb-463c-98a3-1c777ac68897 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:14.423909 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 40/76] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Aug 4 21:02:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:14.424884 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b80c5-4e85-4b45-9979-6918626f680a None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:14.425172 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b80c5-4e85-4b45-9979-6918626f680a None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:14.425489 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b80c5-4e85-4b45-9979-6918626f680a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:14.442887 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-d10b80c5-4e85-4b45-9979-6918626f680a None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:14.444094 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-d10b80c5-4e85-4b45-9979-6918626f680a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:14.463101 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dde70c4-1bc8-4e63-acec-e45ce4c42511 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:14.491416 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:14.497375 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcb5e83-1832-46ba-8c6a-e28182ad4fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:14.498003 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcb5e83-1832-46ba-8c6a-e28182ad4fb8 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:14.498359 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcb5e83-1832-46ba-8c6a-e28182ad4fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:14.498637 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcb5e83-1832-46ba-8c6a-e28182ad4fb8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:14.509103 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 41/77] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:02:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:02:14.509907 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10b80c5-4e85-4b45-9979-6918626f680a None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:14.574077 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcb5e83-1832-46ba-8c6a-e28182ad4fb8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:14.575619 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcb5e83-1832-46ba-8c6a-e28182ad4fb8 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:14.584437 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:14.608512 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10b80c5-4e85-4b45-9979-6918626f680a None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:14.642584 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eddaf89-b595-42ec-8cf4-63ef85473f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:14.643148 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eddaf89-b595-42ec-8cf4-63ef85473f67 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:14.643470 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eddaf89-b595-42ec-8cf4-63ef85473f67 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:14.643750 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eddaf89-b595-42ec-8cf4-63ef85473f67 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:14.654074 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eddaf89-b595-42ec-8cf4-63ef85473f67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:02:14.659230 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-3eddaf89-b595-42ec-8cf4-63ef85473f67 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 21:02:14.659230 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:02:14.659230 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:02:14.659230 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:02:14.659230 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:02:14.659230 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:02:14.659230 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:02:14.659230 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:02:14.659230 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:02:14.659230 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:02:14.659230 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:02:14.659230 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:02:14.659230 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:02:14.659230 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:02:14.659230 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:02:14.659230 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:02:14.660665 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:02:14.660665 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:02:14.660665 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:02:14.660665 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:02:14.660665 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:02:14.660665 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:02:14.660665 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:02:14.660665 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:02:14.660665 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:02:14.660665 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:02:14.660665 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:02:14.660665 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:02:14.660665 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:02:14.660665 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:02:14.660665 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:02:14.660665 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:02:14.661406 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:02:14.661406 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:02:14.661406 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:02:14.661406 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:02:14.661406 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:02:14.661406 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:02:14.661406 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:02:14.661406 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:02:14.661406 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:02:14.661406 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:02:14.661406 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 21:02:14.661406 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:02:14.661406 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 42/78] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Aug 4 21:02:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:02:14.662035 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eddaf89-b595-42ec-8cf4-63ef85473f67 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:14.682336 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 37/79] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Aug 4 21:02:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 21:02:14.682336 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcb5e83-1832-46ba-8c6a-e28182ad4fb8 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:14.712054 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:14.767327 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eddaf89-b595-42ec-8cf4-63ef85473f67 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:14.774873 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcb5e83-1832-46ba-8c6a-e28182ad4fb8 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:14.782672 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:14.801610 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d98de19-5b6d-4f92-98bb-12d9a53a80c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:14.802238 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d98de19-5b6d-4f92-98bb-12d9a53a80c3 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:14.802528 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d98de19-5b6d-4f92-98bb-12d9a53a80c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:14.802798 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d98de19-5b6d-4f92-98bb-12d9a53a80c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:14.803587 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d98de19-5b6d-4f92-98bb-12d9a53a80c3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:14.803899 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d98de19-5b6d-4f92-98bb-12d9a53a80c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:14.805310 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d98de19-5b6d-4f92-98bb-12d9a53a80c3 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:14.809400 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4abc549-35a9-4cf8-8db6-e4fbde77bf0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:14.810054 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4abc549-35a9-4cf8-8db6-e4fbde77bf0b None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:14.810343 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4abc549-35a9-4cf8-8db6-e4fbde77bf0b None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:14.810655 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4abc549-35a9-4cf8-8db6-e4fbde77bf0b None admin] PATH_INFO: `/v3/users/bc6bad33dfb24427b1ccf30de722b45a` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:14.811986 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4abc549-35a9-4cf8-8db6-e4fbde77bf0b None admin] RBAC: Authorizing `identity:get_user(user_id=bc6bad33dfb24427b1ccf30de722b45a)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:14.813365 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 43/80] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Aug 4 21:02:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:14.813688 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4abc549-35a9-4cf8-8db6-e4fbde77bf0b None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:14.814166 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d98de19-5b6d-4f92-98bb-12d9a53a80c3 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:14.815947 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 38/81] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Aug 4 21:02:14 2022] GET /identity/v3/users/bc6bad33dfb24427b1ccf30de722b45a => generated 247 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:14.816705 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4abc549-35a9-4cf8-8db6-e4fbde77bf0b None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:14.912441 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d98de19-5b6d-4f92-98bb-12d9a53a80c3 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:14.915526 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4abc549-35a9-4cf8-8db6-e4fbde77bf0b None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:14.919338 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:14.945063 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869bdc20-e662-4321-ad59-52088f53ed5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:14.945711 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869bdc20-e662-4321-ad59-52088f53ed5e None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:14.945996 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869bdc20-e662-4321-ad59-52088f53ed5e None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:14.946298 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869bdc20-e662-4321-ad59-52088f53ed5e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:14.947470 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869bdc20-e662-4321-ad59-52088f53ed5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:14.948961 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869bdc20-e662-4321-ad59-52088f53ed5e None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:14.950739 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 44/82] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:02:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:14.951403 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869bdc20-e662-4321-ad59-52088f53ed5e None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:14.953335 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1535e7-ce92-4ff9-8cc1-06447ce40961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:14.954030 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1535e7-ce92-4ff9-8cc1-06447ce40961 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:14.954321 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1535e7-ce92-4ff9-8cc1-06447ce40961 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:14.954594 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1535e7-ce92-4ff9-8cc1-06447ce40961 None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:14.955625 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1535e7-ce92-4ff9-8cc1-06447ce40961 None admin] RBAC: Adding query filter params (name=member) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:14.955959 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1535e7-ce92-4ff9-8cc1-06447ce40961 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:14.957852 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1535e7-ce92-4ff9-8cc1-06447ce40961 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:14.966253 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 39/83] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Aug 4 21:02:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:14.967056 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1535e7-ce92-4ff9-8cc1-06447ce40961 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:15.005902 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:15.053275 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869bdc20-e662-4321-ad59-52088f53ed5e None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:15.070293 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1535e7-ce92-4ff9-8cc1-06447ce40961 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:15.087128 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca35095c-218e-48fb-9ca3-b2b89a170edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:15.087799 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca35095c-218e-48fb-9ca3-b2b89a170edf None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:15.088081 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca35095c-218e-48fb-9ca3-b2b89a170edf None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:15.088354 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca35095c-218e-48fb-9ca3-b2b89a170edf None admin] PATH_INFO: `/v3/projects/b8f804a236884f039371233cce01ecef` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:15.089494 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca35095c-218e-48fb-9ca3-b2b89a170edf None admin] RBAC: Authorizing `identity:get_project(project_id=b8f804a236884f039371233cce01ecef)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:15.090962 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca35095c-218e-48fb-9ca3-b2b89a170edf None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:15.092645 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 45/84] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Aug 4 21:02:14 2022] GET /identity/v3/projects/b8f804a236884f039371233cce01ecef => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:15.093486 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca35095c-218e-48fb-9ca3-b2b89a170edf None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:15.104844 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8419dbb5-f02b-4919-b566-ca67ea39e2d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:15.105491 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8419dbb5-f02b-4919-b566-ca67ea39e2d9 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:15.105778 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8419dbb5-f02b-4919-b566-ca67ea39e2d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:15.106055 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8419dbb5-f02b-4919-b566-ca67ea39e2d9 None admin] PATH_INFO: `/v3/users/d562ab3d5d014f1dbf980dedad3e33c7` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:15.107233 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8419dbb5-f02b-4919-b566-ca67ea39e2d9 None admin] RBAC: Authorizing `identity:get_user(user_id=d562ab3d5d014f1dbf980dedad3e33c7)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:15.108714 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8419dbb5-f02b-4919-b566-ca67ea39e2d9 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:15.110329 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 40/85] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Aug 4 21:02:14 2022] GET /identity/v3/users/d562ab3d5d014f1dbf980dedad3e33c7 => generated 275 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:15.111055 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8419dbb5-f02b-4919-b566-ca67ea39e2d9 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:15.139876 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:15.195987 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca35095c-218e-48fb-9ca3-b2b89a170edf None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:15.209560 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8419dbb5-f02b-4919-b566-ca67ea39e2d9 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:15.211831 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:15.231823 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e9d875-4034-4b22-9ea5-b3501d47a499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:15.232451 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e9d875-4034-4b22-9ea5-b3501d47a499 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:15.232734 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e9d875-4034-4b22-9ea5-b3501d47a499 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:15.233009 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e9d875-4034-4b22-9ea5-b3501d47a499 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:15.245160 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e1a229-50fa-4074-a231-7cfa80dc0f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:15.245978 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e1a229-50fa-4074-a231-7cfa80dc0f20 None admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:15.246430 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e1a229-50fa-4074-a231-7cfa80dc0f20 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:15.247004 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e1a229-50fa-4074-a231-7cfa80dc0f20 None admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:15.247740 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e1a229-50fa-4074-a231-7cfa80dc0f20 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:15.249210 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e1a229-50fa-4074-a231-7cfa80dc0f20 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:15.306349 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e9d875-4034-4b22-9ea5-b3501d47a499 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:15.307922 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e9d875-4034-4b22-9ea5-b3501d47a499 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:15.348867 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:15.355025 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 41/86] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Aug 4 21:02:15 2022] POST /identity/v3/users => generated 289 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:02:15.355892 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e1a229-50fa-4074-a231-7cfa80dc0f20 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:15.421113 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 46/87] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Aug 4 21:02:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 21:02:15.427276 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e9d875-4034-4b22-9ea5-b3501d47a499 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:15.471337 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e1a229-50fa-4074-a231-7cfa80dc0f20 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:15.509247 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ac89b5-5dec-4697-ae04-41e93a9a608a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:15.509992 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ac89b5-5dec-4697-ae04-41e93a9a608a None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:15.510184 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ac89b5-5dec-4697-ae04-41e93a9a608a None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:15.510462 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ac89b5-5dec-4697-ae04-41e93a9a608a None admin] PATH_INFO: `/v3/users/d562ab3d5d014f1dbf980dedad3e33c7` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:15.511719 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ac89b5-5dec-4697-ae04-41e93a9a608a None admin] RBAC: Authorizing `identity:get_user(user_id=d562ab3d5d014f1dbf980dedad3e33c7)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:15.513192 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ac89b5-5dec-4697-ae04-41e93a9a608a None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:15.514858 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 42/88] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Aug 4 21:02:15 2022] GET /identity/v3/users/d562ab3d5d014f1dbf980dedad3e33c7 => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:15.515651 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ac89b5-5dec-4697-ae04-41e93a9a608a None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:15.549044 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:15.551138 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e9d875-4034-4b22-9ea5-b3501d47a499 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:15.593344 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627ae852-0a27-4494-9e80-0b9e23e52332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:15.594009 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627ae852-0a27-4494-9e80-0b9e23e52332 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:15.594298 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627ae852-0a27-4494-9e80-0b9e23e52332 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:15.594584 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627ae852-0a27-4494-9e80-0b9e23e52332 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:15.596611 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627ae852-0a27-4494-9e80-0b9e23e52332 None admin] RBAC: Adding query filter params (role.id=8feebc60363c4feb870718b1721e5324, scope.project.id=b8f804a236884f039371233cce01ecef, user.id=d562ab3d5d014f1dbf980dedad3e33c7) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:15.596611 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627ae852-0a27-4494-9e80-0b9e23e52332 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:15.598096 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627ae852-0a27-4494-9e80-0b9e23e52332 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:15.604754 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 47/89] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Aug 4 21:02:15 2022] GET /identity/v3/role_assignments?user.id=d562ab3d5d014f1dbf980dedad3e33c7&scope.project.id=b8f804a236884f039371233cce01ecef&role.id=8feebc60363c4feb870718b1721e5324 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:15.606084 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627ae852-0a27-4494-9e80-0b9e23e52332 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:15.617696 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:15.648645 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ac89b5-5dec-4697-ae04-41e93a9a608a None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:15.694944 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36203675-2782-4e45-affb-fffc69669ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:15.695627 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36203675-2782-4e45-affb-fffc69669ee9 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:15.695909 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36203675-2782-4e45-affb-fffc69669ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:15.696184 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36203675-2782-4e45-affb-fffc69669ee9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:15.761278 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627ae852-0a27-4494-9e80-0b9e23e52332 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:15.800096 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36203675-2782-4e45-affb-fffc69669ee9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:15.802354 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36203675-2782-4e45-affb-fffc69669ee9 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:15.802845 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece3a849-e477-4054-aa9f-d3da4603e2bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:15.803586 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece3a849-e477-4054-aa9f-d3da4603e2bd None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:15.803883 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece3a849-e477-4054-aa9f-d3da4603e2bd None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:15.804212 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece3a849-e477-4054-aa9f-d3da4603e2bd None admin] PATH_INFO: `/v3/projects/b8f804a236884f039371233cce01ecef` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:15.805421 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece3a849-e477-4054-aa9f-d3da4603e2bd None admin] RBAC: Authorizing `identity:get_project(project_id=b8f804a236884f039371233cce01ecef)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:15.806957 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece3a849-e477-4054-aa9f-d3da4603e2bd None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:15.809044 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 48/90] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Aug 4 21:02:15 2022] GET /identity/v3/projects/b8f804a236884f039371233cce01ecef => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:15.809844 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece3a849-e477-4054-aa9f-d3da4603e2bd None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:15.837544 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:15.921979 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:15.969861 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 43/91] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Aug 4 21:02:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 21:02:15.970722 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36203675-2782-4e45-affb-fffc69669ee9 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:15.975254 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece3a849-e477-4054-aa9f-d3da4603e2bd None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:16.015202 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e5da2d-3ee5-4651-a31d-e111dceb06bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:16.015798 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e5da2d-3ee5-4651-a31d-e111dceb06bf None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:16.016088 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e5da2d-3ee5-4651-a31d-e111dceb06bf None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:16.016369 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e5da2d-3ee5-4651-a31d-e111dceb06bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:16.155502 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36203675-2782-4e45-affb-fffc69669ee9 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:16.168238 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e5da2d-3ee5-4651-a31d-e111dceb06bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:16.169815 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e5da2d-3ee5-4651-a31d-e111dceb06bf None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:16.209980 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:16.252094 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65937ad-3a63-4b76-a613-d29789c9c6b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:16.252744 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65937ad-3a63-4b76-a613-d29789c9c6b1 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:16.253032 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65937ad-3a63-4b76-a613-d29789c9c6b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:16.253311 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65937ad-3a63-4b76-a613-d29789c9c6b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:16.254680 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65937ad-3a63-4b76-a613-d29789c9c6b1 None admin] RBAC: Adding query filter params (role.id=a432dd4735944ecfab2fa5ce1705d356, scope.project.id=b8f804a236884f039371233cce01ecef, user.id=bc6bad33dfb24427b1ccf30de722b45a) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:16.254995 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65937ad-3a63-4b76-a613-d29789c9c6b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:16.256474 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65937ad-3a63-4b76-a613-d29789c9c6b1 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:16.258222 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:16.270309 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 44/92] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Aug 4 21:02:15 2022] GET /identity/v3/role_assignments?user.id=bc6bad33dfb24427b1ccf30de722b45a&scope.project.id=b8f804a236884f039371233cce01ecef&role.id=a432dd4735944ecfab2fa5ce1705d356 => generated 259 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:16.271067 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65937ad-3a63-4b76-a613-d29789c9c6b1 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:16.308538 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 49/93] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Aug 4 21:02:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 21:02:16.314697 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e5da2d-3ee5-4651-a31d-e111dceb06bf None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:16.464122 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65937ad-3a63-4b76-a613-d29789c9c6b1 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:16.529756 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e5da2d-3ee5-4651-a31d-e111dceb06bf None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:16.530385 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7deb65-d2e3-4f46-a647-8b271a86c6ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:16.530999 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7deb65-d2e3-4f46-a647-8b271a86c6ad None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:16.531684 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7deb65-d2e3-4f46-a647-8b271a86c6ad None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:16.531978 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7deb65-d2e3-4f46-a647-8b271a86c6ad None admin] PATH_INFO: `/v3/projects/b8f804a236884f039371233cce01ecef` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:16.536124 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7deb65-d2e3-4f46-a647-8b271a86c6ad None admin] RBAC: Authorizing `identity:get_project(project_id=b8f804a236884f039371233cce01ecef)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:16.538896 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7deb65-d2e3-4f46-a647-8b271a86c6ad None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:16.541071 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 45/94] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Aug 4 21:02:16 2022] GET /identity/v3/projects/b8f804a236884f039371233cce01ecef => generated 298 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:16.547304 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7deb65-d2e3-4f46-a647-8b271a86c6ad None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:16.596959 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8fbc3f-12e9-402d-aa7d-1b2a47fdcda7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:16.598163 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8fbc3f-12e9-402d-aa7d-1b2a47fdcda7 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:16.598451 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8fbc3f-12e9-402d-aa7d-1b2a47fdcda7 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:16.598730 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8fbc3f-12e9-402d-aa7d-1b2a47fdcda7 None admin] PATH_INFO: `/v3/projects/55db8128e376436dbc43a52611f93a5f` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:16.604859 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8fbc3f-12e9-402d-aa7d-1b2a47fdcda7 None admin] RBAC: Authorizing `identity:get_project(project_id=55db8128e376436dbc43a52611f93a5f)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:16.606371 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8fbc3f-12e9-402d-aa7d-1b2a47fdcda7 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:16.614574 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 50/95] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Aug 4 21:02:16 2022] GET /identity/v3/projects/55db8128e376436dbc43a52611f93a5f => generated 312 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:16.615449 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8fbc3f-12e9-402d-aa7d-1b2a47fdcda7 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:16.659214 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:16.705395 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:16.729073 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7deb65-d2e3-4f46-a647-8b271a86c6ad None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:16.793468 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea66437-0964-4b93-8189-650af0f03b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:16.794016 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea66437-0964-4b93-8189-650af0f03b0c None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:16.794305 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea66437-0964-4b93-8189-650af0f03b0c None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:16.794581 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea66437-0964-4b93-8189-650af0f03b0c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:16.795404 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea66437-0964-4b93-8189-650af0f03b0c None admin] RBAC: Adding query filter params (role.id=70587326d8584316b73b380df2ae8e67, scope.project.id=b8f804a236884f039371233cce01ecef, user.id=d562ab3d5d014f1dbf980dedad3e33c7) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:16.795719 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea66437-0964-4b93-8189-650af0f03b0c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:16.797183 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea66437-0964-4b93-8189-650af0f03b0c None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:16.807474 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 46/96] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Aug 4 21:02:16 2022] GET /identity/v3/role_assignments?user.id=d562ab3d5d014f1dbf980dedad3e33c7&scope.project.id=b8f804a236884f039371233cce01ecef&role.id=70587326d8584316b73b380df2ae8e67 => generated 259 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:16.850140 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8fbc3f-12e9-402d-aa7d-1b2a47fdcda7 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:16.906425 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bf23d7-8ba2-483f-bb25-0e7b4ea46241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:16.907457 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bf23d7-8ba2-483f-bb25-0e7b4ea46241 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:16.908200 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bf23d7-8ba2-483f-bb25-0e7b4ea46241 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:16.908697 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bf23d7-8ba2-483f-bb25-0e7b4ea46241 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:16.909780 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bf23d7-8ba2-483f-bb25-0e7b4ea46241 None admin] RBAC: Adding query filter params (role.id=8feebc60363c4feb870718b1721e5324, scope.project.id=55db8128e376436dbc43a52611f93a5f, user.id=d562ab3d5d014f1dbf980dedad3e33c7) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:16.910822 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bf23d7-8ba2-483f-bb25-0e7b4ea46241 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:16.912589 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bf23d7-8ba2-483f-bb25-0e7b4ea46241 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:16.923595 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 51/97] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Aug 4 21:02:16 2022] GET /identity/v3/role_assignments?user.id=d562ab3d5d014f1dbf980dedad3e33c7&scope.project.id=55db8128e376436dbc43a52611f93a5f&role.id=8feebc60363c4feb870718b1721e5324 => generated 259 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:18.037856 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eced2f-6b3e-4853-8ec9-6398745883d1 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:18.038393 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eced2f-6b3e-4853-8ec9-6398745883d1 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:18.038393 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eced2f-6b3e-4853-8ec9-6398745883d1 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:18.039893 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 47/98] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:02:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:02:18.045525 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9e188d-bc8b-440d-bd6d-52e216fbebdd None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:18.045959 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9e188d-bc8b-440d-bd6d-52e216fbebdd None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:18.046170 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9e188d-bc8b-440d-bd6d-52e216fbebdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:18.068964 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-dc9e188d-bc8b-440d-bd6d-52e216fbebdd None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:18.070175 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-dc9e188d-bc8b-440d-bd6d-52e216fbebdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:18.147844 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 52/99] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:02:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:02:18.184088 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eced2f-6b3e-4853-8ec9-6398745883d1 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:18.299177 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eced2f-6b3e-4853-8ec9-6398745883d1 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:18.323269 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:18.335439 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2f0667-1778-4c27-92a4-b78ef057c4b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:18.335786 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2f0667-1778-4c27-92a4-b78ef057c4b6 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:18.336098 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2f0667-1778-4c27-92a4-b78ef057c4b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:18.336380 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2f0667-1778-4c27-92a4-b78ef057c4b6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:18.348445 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2f0667-1778-4c27-92a4-b78ef057c4b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:02:18.353913 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-6a2f0667-1778-4c27-92a4-b78ef057c4b6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 21:02:18.353913 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:02:18.353913 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:02:18.353913 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:02:18.353913 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:02:18.353913 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:02:18.353913 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:02:18.353913 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:02:18.353913 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:02:18.353913 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:02:18.353913 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:02:18.353913 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:02:18.353913 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:02:18.353913 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:02:18.353913 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:02:18.353913 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:02:18.354786 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:02:18.354786 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:02:18.354786 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:02:18.354786 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:02:18.354786 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:02:18.354786 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:02:18.354786 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:02:18.354786 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:02:18.354786 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:02:18.354786 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:02:18.354786 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:02:18.354786 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:02:18.354786 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:02:18.354786 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:02:18.354786 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:02:18.354786 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:02:18.356155 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:02:18.356155 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:02:18.356155 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:02:18.356155 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:02:18.356155 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:02:18.356155 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:02:18.356155 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:02:18.356155 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:02:18.356155 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:02:18.356155 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:02:18.356155 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 21:02:18.356155 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:02:18.356155 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 48/100] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Aug 4 21:02:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:02:18.361064 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9e188d-bc8b-440d-bd6d-52e216fbebdd None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:18.400426 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:18.425187 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce1eed-ca6f-4401-aa03-fcba3e34c3cf None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:18.425499 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce1eed-ca6f-4401-aa03-fcba3e34c3cf None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:18.425788 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce1eed-ca6f-4401-aa03-fcba3e34c3cf None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:18.427155 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 49/101] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:02:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 21:02:18.432075 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6911a7d9-8114-4222-9c98-a6454243371f None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:18.432370 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6911a7d9-8114-4222-9c98-a6454243371f None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:18.432650 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6911a7d9-8114-4222-9c98-a6454243371f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:18.452254 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-6911a7d9-8114-4222-9c98-a6454243371f None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:18.453571 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-6911a7d9-8114-4222-9c98-a6454243371f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:18.504952 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9e188d-bc8b-440d-bd6d-52e216fbebdd None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:18.561408 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 50/102] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:02:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:02:18.564321 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7817b37-e732-463b-a881-7e0c357022d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:18.564960 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7817b37-e732-463b-a881-7e0c357022d7 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:18.565249 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7817b37-e732-463b-a881-7e0c357022d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:18.565584 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7817b37-e732-463b-a881-7e0c357022d7 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:18.566360 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7817b37-e732-463b-a881-7e0c357022d7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:18.566673 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7817b37-e732-463b-a881-7e0c357022d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:18.568184 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7817b37-e732-463b-a881-7e0c357022d7 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:18.585633 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 53/103] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Aug 4 21:02:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:18.586799 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6911a7d9-8114-4222-9c98-a6454243371f None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:18.591787 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7817b37-e732-463b-a881-7e0c357022d7 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:18.736983 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:18.741829 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7817b37-e732-463b-a881-7e0c357022d7 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:18.767069 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6911a7d9-8114-4222-9c98-a6454243371f None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:18.784491 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd381c6-3a32-46a1-a450-9cadf3fcbff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:18.785696 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd381c6-3a32-46a1-a450-9cadf3fcbff9 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:18.785984 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd381c6-3a32-46a1-a450-9cadf3fcbff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:18.786263 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd381c6-3a32-46a1-a450-9cadf3fcbff9 None admin] PATH_INFO: `/v3/users/40bb688e6e224290b1de6e4873bb7143` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:18.791183 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:18.803512 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd381c6-3a32-46a1-a450-9cadf3fcbff9 None admin] RBAC: Authorizing `identity:get_user(user_id=40bb688e6e224290b1de6e4873bb7143)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:18.805375 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd381c6-3a32-46a1-a450-9cadf3fcbff9 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:18.807268 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 54/104] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Aug 4 21:02:18 2022] GET /identity/v3/users/40bb688e6e224290b1de6e4873bb7143 => generated 289 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:18.808384 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958330b8-f730-42b2-8f25-4f9651a54eda None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:18.808677 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958330b8-f730-42b2-8f25-4f9651a54eda None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:18.808957 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958330b8-f730-42b2-8f25-4f9651a54eda None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:18.810692 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 55/105] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:02:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:02:18.815664 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958330b8-f730-42b2-8f25-4f9651a54eda None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:18.847737 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f10b87a-c073-43f2-ba1f-33619ba9fda8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:18.848406 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f10b87a-c073-43f2-ba1f-33619ba9fda8 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:18.848709 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f10b87a-c073-43f2-ba1f-33619ba9fda8 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:18.848974 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f10b87a-c073-43f2-ba1f-33619ba9fda8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:18.856240 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f10b87a-c073-43f2-ba1f-33619ba9fda8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:18.857738 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f10b87a-c073-43f2-ba1f-33619ba9fda8 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:18.865481 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 51/106] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:02:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:18.866770 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d260335d-4b2f-443a-913b-47dfa0ddd23e None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:18.867067 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d260335d-4b2f-443a-913b-47dfa0ddd23e None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:18.869058 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d260335d-4b2f-443a-913b-47dfa0ddd23e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:18.901299 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-d260335d-4b2f-443a-913b-47dfa0ddd23e None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:18.902502 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-d260335d-4b2f-443a-913b-47dfa0ddd23e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:18.975432 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 52/107] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:02:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:02:18.975432 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d260335d-4b2f-443a-913b-47dfa0ddd23e None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:18.988947 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958330b8-f730-42b2-8f25-4f9651a54eda None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:19.106925 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d260335d-4b2f-443a-913b-47dfa0ddd23e None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:19.118863 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:19.140739 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a8ffee-66d3-40e4-8ebe-6a9730d1f470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:19.141371 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a8ffee-66d3-40e4-8ebe-6a9730d1f470 None admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:19.141659 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a8ffee-66d3-40e4-8ebe-6a9730d1f470 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:19.141940 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a8ffee-66d3-40e4-8ebe-6a9730d1f470 None admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:19.142842 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a8ffee-66d3-40e4-8ebe-6a9730d1f470 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:19.145069 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a8ffee-66d3-40e4-8ebe-6a9730d1f470 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:19.148536 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed4f410-504d-488f-8775-c7edde66a64b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:19.149731 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed4f410-504d-488f-8775-c7edde66a64b None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:19.150220 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed4f410-504d-488f-8775-c7edde66a64b None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:19.150719 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed4f410-504d-488f-8775-c7edde66a64b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:19.195695 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:19.242691 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 53/108] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Aug 4 21:02:18 2022] POST /identity/v3/projects => generated 302 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:02:19.243432 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a8ffee-66d3-40e4-8ebe-6a9730d1f470 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:19.254401 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed4f410-504d-488f-8775-c7edde66a64b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:19.255959 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed4f410-504d-488f-8775-c7edde66a64b None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:19.413138 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a8ffee-66d3-40e4-8ebe-6a9730d1f470 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:19.469219 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 56/109] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Aug 4 21:02:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 654 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 21:02:19.470505 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6000b0b8-e521-4f3c-a488-4b10ab0cd367 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:19.471790 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6000b0b8-e521-4f3c-a488-4b10ab0cd367 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:19.471790 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6000b0b8-e521-4f3c-a488-4b10ab0cd367 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:19.472742 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 57/110] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:02:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:02:19.476060 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6000b0b8-e521-4f3c-a488-4b10ab0cd367 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:19.483432 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d0c429-8a52-459e-9f64-3f3285366aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:19.484367 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d0c429-8a52-459e-9f64-3f3285366aec None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:19.484864 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d0c429-8a52-459e-9f64-3f3285366aec None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:19.485328 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d0c429-8a52-459e-9f64-3f3285366aec None admin] PATH_INFO: `/v3/roles/member` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:19.500396 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d0c429-8a52-459e-9f64-3f3285366aec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:02:19.507816 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-40d0c429-8a52-459e-9f64-3f3285366aec None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 21:02:19.507816 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:02:19.507816 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:02:19.507816 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:02:19.507816 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:02:19.507816 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:02:19.507816 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:02:19.507816 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:02:19.507816 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:02:19.507816 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:02:19.507816 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:02:19.507816 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:02:19.507816 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:02:19.507816 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:02:19.507816 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:02:19.507816 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:02:19.508711 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:02:19.508711 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:02:19.508711 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:02:19.508711 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:02:19.508711 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:02:19.508711 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:02:19.508711 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:02:19.508711 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:02:19.508711 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:02:19.508711 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:02:19.508711 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:02:19.508711 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:02:19.508711 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:02:19.508711 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:02:19.508711 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:02:19.508711 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:02:19.509453 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:02:19.509453 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:02:19.509453 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:02:19.509453 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:02:19.509453 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:02:19.509453 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:02:19.509453 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:02:19.509453 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:02:19.509453 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:02:19.509453 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:02:19.509453 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 21:02:19.509453 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:02:19.512468 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 54/111] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Aug 4 21:02:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:02:19.514220 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e2a23f-a453-4aa0-9644-ceea4107934f None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:19.514706 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e2a23f-a453-4aa0-9644-ceea4107934f None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:19.515172 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e2a23f-a453-4aa0-9644-ceea4107934f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:19.540350 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-e0e2a23f-a453-4aa0-9644-ceea4107934f None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:19.542759 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-e0e2a23f-a453-4aa0-9644-ceea4107934f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:19.555632 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:19.568666 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:19.608981 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6000b0b8-e521-4f3c-a488-4b10ab0cd367 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:19.641987 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 55/112] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:02:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:02:19.643020 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e2a23f-a453-4aa0-9644-ceea4107934f None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:19.648258 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3639528c-0193-4eda-90b5-478b8c85771e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:19.648991 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3639528c-0193-4eda-90b5-478b8c85771e None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:19.649282 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3639528c-0193-4eda-90b5-478b8c85771e None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:19.649560 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3639528c-0193-4eda-90b5-478b8c85771e None admin] PATH_INFO: `/v3/projects/b8f804a236884f039371233cce01ecef` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:19.650785 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3639528c-0193-4eda-90b5-478b8c85771e None admin] RBAC: Authorizing `identity:get_project(project_id=b8f804a236884f039371233cce01ecef)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:19.652338 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3639528c-0193-4eda-90b5-478b8c85771e None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:19.654611 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 58/113] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Aug 4 21:02:19 2022] GET /identity/v3/projects/b8f804a236884f039371233cce01ecef => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:19.655986 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc4d97a-ecb9-4096-9b2a-122fa1550677 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:19.656281 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc4d97a-ecb9-4096-9b2a-122fa1550677 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:19.656563 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc4d97a-ecb9-4096-9b2a-122fa1550677 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:19.657836 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 59/114] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:02:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:02:19.664572 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc4d97a-ecb9-4096-9b2a-122fa1550677 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:19.762088 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e2a23f-a453-4aa0-9644-ceea4107934f None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:19.772805 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc4d97a-ecb9-4096-9b2a-122fa1550677 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:19.797376 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:19.801352 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c6cab3-384a-42e9-958c-ec0225781004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:19.801989 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c6cab3-384a-42e9-958c-ec0225781004 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:19.802270 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c6cab3-384a-42e9-958c-ec0225781004 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:19.802544 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c6cab3-384a-42e9-958c-ec0225781004 None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:19.803268 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c6cab3-384a-42e9-958c-ec0225781004 None admin] RBAC: Adding query filter params (name=member) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:19.803603 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c6cab3-384a-42e9-958c-ec0225781004 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:19.805063 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c6cab3-384a-42e9-958c-ec0225781004 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:19.809390 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf52421-e0d1-4dc5-8e9e-31ba843fe95e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:19.810028 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf52421-e0d1-4dc5-8e9e-31ba843fe95e None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:19.810372 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf52421-e0d1-4dc5-8e9e-31ba843fe95e None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:19.810653 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf52421-e0d1-4dc5-8e9e-31ba843fe95e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:19.816928 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf52421-e0d1-4dc5-8e9e-31ba843fe95e None admin] RBAC: Adding query filter params (role.id=5ab0887d24bc41b882f6786b7aab0fe8, scope.project.id=b8f804a236884f039371233cce01ecef, user.id=40bb688e6e224290b1de6e4873bb7143) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:19.816928 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf52421-e0d1-4dc5-8e9e-31ba843fe95e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:19.816928 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf52421-e0d1-4dc5-8e9e-31ba843fe95e None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:19.817139 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 56/115] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Aug 4 21:02:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:19.817139 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67e5a90-07b8-4a91-9b22-52f525495cd7 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:19.817139 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67e5a90-07b8-4a91-9b22-52f525495cd7 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:19.817325 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67e5a90-07b8-4a91-9b22-52f525495cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:19.820656 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 60/116] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Aug 4 21:02:19 2022] GET /identity/v3/role_assignments?user.id=40bb688e6e224290b1de6e4873bb7143&scope.project.id=b8f804a236884f039371233cce01ecef&role.id=5ab0887d24bc41b882f6786b7aab0fe8 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:02:19.821559 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf52421-e0d1-4dc5-8e9e-31ba843fe95e None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:19.838369 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-d67e5a90-07b8-4a91-9b22-52f525495cd7 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:19.839763 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-d67e5a90-07b8-4a91-9b22-52f525495cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:19.852274 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:19.917860 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 57/117] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:02:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:02:19.918649 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67e5a90-07b8-4a91-9b22-52f525495cd7 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:19.950454 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf52421-e0d1-4dc5-8e9e-31ba843fe95e None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:20.034033 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67e5a90-07b8-4a91-9b22-52f525495cd7 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:20.036746 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f3db26-bc8d-4ccd-9e13-a4a3593977ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:20.037423 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f3db26-bc8d-4ccd-9e13-a4a3593977ef None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:20.037712 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f3db26-bc8d-4ccd-9e13-a4a3593977ef None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:20.037989 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f3db26-bc8d-4ccd-9e13-a4a3593977ef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:20.072395 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f3db26-bc8d-4ccd-9e13-a4a3593977ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:02:20.073165 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5778b56-bb00-4304-8ba3-ab137706c4ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:20.073872 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5778b56-bb00-4304-8ba3-ab137706c4ce None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:20.074149 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5778b56-bb00-4304-8ba3-ab137706c4ce None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:20.074424 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5778b56-bb00-4304-8ba3-ab137706c4ce None admin] PATH_INFO: `/v3/users/d562ab3d5d014f1dbf980dedad3e33c7` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:20.075634 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5778b56-bb00-4304-8ba3-ab137706c4ce None admin] RBAC: Authorizing `identity:get_user(user_id=d562ab3d5d014f1dbf980dedad3e33c7)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:20.077099 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5778b56-bb00-4304-8ba3-ab137706c4ce None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:20.078678 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-29f3db26-bc8d-4ccd-9e13-a4a3593977ef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:02:20.078678 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:02:20.078678 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:02:20.078678 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:02:20.078678 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:02:20.078678 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:02:20.078678 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:02:20.078678 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:02:20.078678 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:02:20.078678 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:02:20.078678 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:02:20.078678 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:02:20.078678 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:02:20.078678 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:02:20.078678 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:02:20.078678 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:02:20.079647 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:02:20.079647 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:02:20.079647 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:02:20.079647 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:02:20.079647 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:02:20.079647 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:02:20.079647 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:02:20.079647 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:02:20.079647 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:02:20.079647 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:02:20.079647 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:02:20.079647 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:02:20.079647 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:02:20.079647 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:02:20.079647 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:02:20.079647 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:02:20.080423 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:02:20.080423 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:02:20.080423 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:02:20.080423 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:02:20.080423 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:02:20.080423 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:02:20.080423 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:02:20.080423 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:02:20.080423 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:02:20.080423 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:02:20.080423 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:02:20.080423 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:02:20.080997 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 58/118] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Aug 4 21:02:19 2022] GET /identity/v3/users/d562ab3d5d014f1dbf980dedad3e33c7 => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:20.080997 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5778b56-bb00-4304-8ba3-ab137706c4ce None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:20.085054 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 61/119] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Aug 4 21:02:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:02:20.090518 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f3db26-bc8d-4ccd-9e13-a4a3593977ef None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:20.098511 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:20.154793 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:20.212161 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5778b56-bb00-4304-8ba3-ab137706c4ce None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:20.214657 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f3db26-bc8d-4ccd-9e13-a4a3593977ef None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:20.256497 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdf0d2f-c9a7-41b7-a4a6-06732a2063b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:20.257270 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdf0d2f-c9a7-41b7-a4a6-06732a2063b8 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:20.257627 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdf0d2f-c9a7-41b7-a4a6-06732a2063b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:20.258001 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdf0d2f-c9a7-41b7-a4a6-06732a2063b8 None admin] PATH_INFO: `/v3/projects/b8f804a236884f039371233cce01ecef` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:20.258164 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed67b91-8b08-4e0e-8272-c9643a24e8c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:20.258817 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed67b91-8b08-4e0e-8272-c9643a24e8c2 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:20.259103 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed67b91-8b08-4e0e-8272-c9643a24e8c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:20.259411 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed67b91-8b08-4e0e-8272-c9643a24e8c2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:20.259522 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdf0d2f-c9a7-41b7-a4a6-06732a2063b8 None admin] RBAC: Authorizing `identity:get_project(project_id=b8f804a236884f039371233cce01ecef)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:20.261246 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdf0d2f-c9a7-41b7-a4a6-06732a2063b8 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:20.263456 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 62/120] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Aug 4 21:02:20 2022] GET /identity/v3/projects/b8f804a236884f039371233cce01ecef => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:02:20.264346 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdf0d2f-c9a7-41b7-a4a6-06732a2063b8 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:20.272736 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed67b91-8b08-4e0e-8272-c9643a24e8c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:02:20.279626 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-6ed67b91-8b08-4e0e-8272-c9643a24e8c2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 21:02:20.279626 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:02:20.279626 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:02:20.279626 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:02:20.279626 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:02:20.279626 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:02:20.279626 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:02:20.279626 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:02:20.279626 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:02:20.279626 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:02:20.279626 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:02:20.279626 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:02:20.279626 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:02:20.279626 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:02:20.279626 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:02:20.279626 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:02:20.280462 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:02:20.280462 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:02:20.280462 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:02:20.280462 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:02:20.280462 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:02:20.280462 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:02:20.280462 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:02:20.280462 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:02:20.280462 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:02:20.280462 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:02:20.280462 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:02:20.280462 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:02:20.280462 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:02:20.280462 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:02:20.280462 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:02:20.280462 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:02:20.281746 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:02:20.281746 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:02:20.281746 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:02:20.281746 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:02:20.281746 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:02:20.281746 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:02:20.281746 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:02:20.281746 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:02:20.281746 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:02:20.281746 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:02:20.281746 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 21:02:20.281746 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:02:20.281746 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 59/121] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Aug 4 21:02:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:02:20.284509 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaca164-f9d3-4764-a874-c9e6eff8c285 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:20.284839 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaca164-f9d3-4764-a874-c9e6eff8c285 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:20.285123 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaca164-f9d3-4764-a874-c9e6eff8c285 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:20.287458 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 60/122] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:02:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:02:20.288461 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaca164-f9d3-4764-a874-c9e6eff8c285 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:20.398095 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdf0d2f-c9a7-41b7-a4a6-06732a2063b8 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:20.400392 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:20.418877 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:20.430072 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaca164-f9d3-4764-a874-c9e6eff8c285 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:20.440608 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef2f29e-e03a-43b3-8b75-a7ca85d8fd3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:20.442639 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef2f29e-e03a-43b3-8b75-a7ca85d8fd3b None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:20.443139 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef2f29e-e03a-43b3-8b75-a7ca85d8fd3b None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:20.444133 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef2f29e-e03a-43b3-8b75-a7ca85d8fd3b None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:20.445898 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef2f29e-e03a-43b3-8b75-a7ca85d8fd3b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:20.446735 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef2f29e-e03a-43b3-8b75-a7ca85d8fd3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:20.448927 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef2f29e-e03a-43b3-8b75-a7ca85d8fd3b None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:20.464606 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 63/123] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Aug 4 21:02:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:20.466046 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef2f29e-e03a-43b3-8b75-a7ca85d8fd3b None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:20.477075 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52de499-ffc6-406a-968f-e656516a7c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:20.477075 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52de499-ffc6-406a-968f-e656516a7c7f None admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:20.477075 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52de499-ffc6-406a-968f-e656516a7c7f None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:20.477075 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52de499-ffc6-406a-968f-e656516a7c7f None admin] PATH_INFO: `/v3/projects/b8f804a236884f039371233cce01ecef/users/d562ab3d5d014f1dbf980dedad3e33c7/roles/8feebc60363c4feb870718b1721e5324` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:20.478688 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52de499-ffc6-406a-968f-e656516a7c7f None admin] RBAC: Authorizing `identity:create_grant(project_id=b8f804a236884f039371233cce01ecef, user_id=d562ab3d5d014f1dbf980dedad3e33c7, role_id=8feebc60363c4feb870718b1721e5324)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:20.480527 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52de499-ffc6-406a-968f-e656516a7c7f None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:20.557671 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 61/124] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Aug 4 21:02:20 2022] PUT /identity/v3/projects/b8f804a236884f039371233cce01ecef/users/d562ab3d5d014f1dbf980dedad3e33c7/roles/8feebc60363c4feb870718b1721e5324 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:02:20.559284 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de767c76-cd3b-4693-8619-90e7f6acb928 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:20.559812 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de767c76-cd3b-4693-8619-90e7f6acb928 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:20.560959 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de767c76-cd3b-4693-8619-90e7f6acb928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:20.576171 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef2f29e-e03a-43b3-8b75-a7ca85d8fd3b None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:20.594033 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-de767c76-cd3b-4693-8619-90e7f6acb928 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:20.599109 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-de767c76-cd3b-4693-8619-90e7f6acb928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:20.662161 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4fb8ee-7f21-488f-8596-6be5ca10ec98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:20.663208 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4fb8ee-7f21-488f-8596-6be5ca10ec98 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:20.663896 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4fb8ee-7f21-488f-8596-6be5ca10ec98 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:20.665112 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4fb8ee-7f21-488f-8596-6be5ca10ec98 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:20.666743 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4fb8ee-7f21-488f-8596-6be5ca10ec98 None admin] RBAC: Adding query filter params (name=member) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:20.667451 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4fb8ee-7f21-488f-8596-6be5ca10ec98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:20.670557 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4fb8ee-7f21-488f-8596-6be5ca10ec98 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:20.693416 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 64/125] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Aug 4 21:02:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:20.694640 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4fb8ee-7f21-488f-8596-6be5ca10ec98 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:20.740039 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 62/126] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:02:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:02:20.741212 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de767c76-cd3b-4693-8619-90e7f6acb928 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:20.780110 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:20.834833 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:20.846630 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4fb8ee-7f21-488f-8596-6be5ca10ec98 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:20.888698 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c89a63f-4dcf-44bf-ace0-d9190825009c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:20.889521 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c89a63f-4dcf-44bf-ace0-d9190825009c None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:20.889922 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c89a63f-4dcf-44bf-ace0-d9190825009c None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:20.890331 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c89a63f-4dcf-44bf-ace0-d9190825009c None admin] PATH_INFO: `/v3/users/bc6bad33dfb24427b1ccf30de722b45a` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:20.890799 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de767c76-cd3b-4693-8619-90e7f6acb928 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:20.892000 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c89a63f-4dcf-44bf-ace0-d9190825009c None admin] RBAC: Authorizing `identity:get_user(user_id=bc6bad33dfb24427b1ccf30de722b45a)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:20.894497 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c89a63f-4dcf-44bf-ace0-d9190825009c None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:20.896483 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 65/127] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Aug 4 21:02:20 2022] GET /identity/v3/users/bc6bad33dfb24427b1ccf30de722b45a => generated 247 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:20.897526 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c89a63f-4dcf-44bf-ace0-d9190825009c None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:20.953283 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a5e819-910c-43c6-bbc5-dace0bab84f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:20.954227 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a5e819-910c-43c6-bbc5-dace0bab84f3 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:20.955550 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a5e819-910c-43c6-bbc5-dace0bab84f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:20.956077 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a5e819-910c-43c6-bbc5-dace0bab84f3 None admin] PATH_INFO: `/v3/users/d562ab3d5d014f1dbf980dedad3e33c7` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:20.959814 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a5e819-910c-43c6-bbc5-dace0bab84f3 None admin] RBAC: Authorizing `identity:get_user(user_id=d562ab3d5d014f1dbf980dedad3e33c7)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:20.961541 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a5e819-910c-43c6-bbc5-dace0bab84f3 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:20.965469 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 63/128] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Aug 4 21:02:20 2022] GET /identity/v3/users/d562ab3d5d014f1dbf980dedad3e33c7 => generated 275 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:20.968321 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a5e819-910c-43c6-bbc5-dace0bab84f3 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:21.096026 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a5e819-910c-43c6-bbc5-dace0bab84f3 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:21.114918 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c89a63f-4dcf-44bf-ace0-d9190825009c None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:21.131755 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:21.134735 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:21.138087 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68d5104-0e70-462a-8b1d-eb72e7162626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:21.139289 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d5104-0e70-462a-8b1d-eb72e7162626 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:21.139785 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d5104-0e70-462a-8b1d-eb72e7162626 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:21.146349 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d5104-0e70-462a-8b1d-eb72e7162626 None admin] PATH_INFO: `/v3/projects/b8f804a236884f039371233cce01ecef` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:21.146349 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68d5104-0e70-462a-8b1d-eb72e7162626 None admin] RBAC: Authorizing `identity:get_project(project_id=b8f804a236884f039371233cce01ecef)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:21.146349 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68d5104-0e70-462a-8b1d-eb72e7162626 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:21.146349 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 64/129] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Aug 4 21:02:20 2022] GET /identity/v3/projects/b8f804a236884f039371233cce01ecef => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:21.146349 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68d5104-0e70-462a-8b1d-eb72e7162626 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:21.191496 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7213215-8a11-4b13-8cce-9595b9f7844a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:21.192899 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7213215-8a11-4b13-8cce-9595b9f7844a None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:21.193433 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7213215-8a11-4b13-8cce-9595b9f7844a None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:21.194816 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7213215-8a11-4b13-8cce-9595b9f7844a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:21.214644 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7213215-8a11-4b13-8cce-9595b9f7844a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:02:21.229971 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-d7213215-8a11-4b13-8cce-9595b9f7844a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 21:02:21.229971 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:02:21.229971 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:02:21.229971 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:02:21.229971 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:02:21.229971 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:02:21.229971 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:02:21.229971 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:02:21.229971 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:02:21.229971 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:02:21.229971 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:02:21.229971 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:02:21.229971 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:02:21.229971 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:02:21.229971 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:02:21.229971 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:02:21.230823 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:02:21.230823 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:02:21.230823 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:02:21.230823 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:02:21.230823 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:02:21.230823 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:02:21.230823 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:02:21.230823 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:02:21.230823 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:02:21.230823 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:02:21.230823 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:02:21.230823 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:02:21.230823 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:02:21.230823 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:02:21.230823 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:02:21.230823 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:02:21.231634 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:02:21.231634 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:02:21.231634 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:02:21.231634 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:02:21.231634 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:02:21.231634 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:02:21.231634 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:02:21.231634 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:02:21.231634 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:02:21.231634 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:02:21.231634 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 21:02:21.231634 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:02:21.233955 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 66/130] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Aug 4 21:02:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:02:21.234934 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7213215-8a11-4b13-8cce-9595b9f7844a None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:21.332244 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68d5104-0e70-462a-8b1d-eb72e7162626 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:21.395975 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82244350-b5e4-490c-af47-abb8d698a8dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:21.396972 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82244350-b5e4-490c-af47-abb8d698a8dc None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:21.397457 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82244350-b5e4-490c-af47-abb8d698a8dc None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:21.397933 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82244350-b5e4-490c-af47-abb8d698a8dc None admin] PATH_INFO: `/v3/projects/55db8128e376436dbc43a52611f93a5f` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:21.399519 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82244350-b5e4-490c-af47-abb8d698a8dc None admin] RBAC: Authorizing `identity:get_project(project_id=55db8128e376436dbc43a52611f93a5f)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:21.401963 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82244350-b5e4-490c-af47-abb8d698a8dc None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:21.406280 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 65/131] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Aug 4 21:02:21 2022] GET /identity/v3/projects/55db8128e376436dbc43a52611f93a5f => generated 312 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:21.408289 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82244350-b5e4-490c-af47-abb8d698a8dc None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:21.417278 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7213215-8a11-4b13-8cce-9595b9f7844a None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:21.476315 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1807454f-c57e-4b70-9c89-4d908a68b56d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:21.477101 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1807454f-c57e-4b70-9c89-4d908a68b56d None admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:21.477392 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1807454f-c57e-4b70-9c89-4d908a68b56d None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:21.477674 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1807454f-c57e-4b70-9c89-4d908a68b56d None admin] PATH_INFO: `/v3/projects/b8f804a236884f039371233cce01ecef/users/bc6bad33dfb24427b1ccf30de722b45a/roles/a432dd4735944ecfab2fa5ce1705d356` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:21.481357 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1807454f-c57e-4b70-9c89-4d908a68b56d None admin] RBAC: Authorizing `identity:create_grant(project_id=b8f804a236884f039371233cce01ecef, user_id=bc6bad33dfb24427b1ccf30de722b45a, role_id=a432dd4735944ecfab2fa5ce1705d356)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:21.482937 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1807454f-c57e-4b70-9c89-4d908a68b56d None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:21.499702 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:21.540957 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82244350-b5e4-490c-af47-abb8d698a8dc None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:21.578290 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd27e62f-a670-4743-b886-792bb5d356fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:21.578918 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd27e62f-a670-4743-b886-792bb5d356fd None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:21.579260 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd27e62f-a670-4743-b886-792bb5d356fd None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:21.579569 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd27e62f-a670-4743-b886-792bb5d356fd None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:21.580287 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd27e62f-a670-4743-b886-792bb5d356fd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:21.580591 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd27e62f-a670-4743-b886-792bb5d356fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:21.582126 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd27e62f-a670-4743-b886-792bb5d356fd None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:21.590842 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 66/132] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Aug 4 21:02:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:21.591654 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd27e62f-a670-4743-b886-792bb5d356fd None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:21.603145 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 67/133] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Aug 4 21:02:21 2022] PUT /identity/v3/projects/b8f804a236884f039371233cce01ecef/users/bc6bad33dfb24427b1ccf30de722b45a/roles/a432dd4735944ecfab2fa5ce1705d356 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:02:21.604029 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1807454f-c57e-4b70-9c89-4d908a68b56d None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:21.686378 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:21.721787 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd27e62f-a670-4743-b886-792bb5d356fd None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:21.787717 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:21.800097 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ab17a7-2526-4253-b740-dabfe9976e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:21.801362 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ab17a7-2526-4253-b740-dabfe9976e78 None admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:21.801647 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ab17a7-2526-4253-b740-dabfe9976e78 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:21.801921 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ab17a7-2526-4253-b740-dabfe9976e78 None admin] PATH_INFO: `/v3/projects/55db8128e376436dbc43a52611f93a5f/users/d562ab3d5d014f1dbf980dedad3e33c7/roles/8feebc60363c4feb870718b1721e5324` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:21.808701 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ab17a7-2526-4253-b740-dabfe9976e78 None admin] RBAC: Authorizing `identity:create_grant(project_id=55db8128e376436dbc43a52611f93a5f, user_id=d562ab3d5d014f1dbf980dedad3e33c7, role_id=8feebc60363c4feb870718b1721e5324)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:21.810282 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ab17a7-2526-4253-b740-dabfe9976e78 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:21.826767 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1807454f-c57e-4b70-9c89-4d908a68b56d None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:21.875938 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a461b43-0e54-48c4-8c24-269ef1c217ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:21.876655 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a461b43-0e54-48c4-8c24-269ef1c217ee None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:21.876941 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a461b43-0e54-48c4-8c24-269ef1c217ee None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:21.877220 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a461b43-0e54-48c4-8c24-269ef1c217ee None admin] PATH_INFO: `/v3/users/d562ab3d5d014f1dbf980dedad3e33c7` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:21.881220 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a461b43-0e54-48c4-8c24-269ef1c217ee None admin] RBAC: Authorizing `identity:get_user(user_id=d562ab3d5d014f1dbf980dedad3e33c7)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:21.882760 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a461b43-0e54-48c4-8c24-269ef1c217ee None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:21.884675 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 68/134] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Aug 4 21:02:21 2022] GET /identity/v3/users/d562ab3d5d014f1dbf980dedad3e33c7 => generated 275 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:21.886371 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 67/135] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Aug 4 21:02:21 2022] PUT /identity/v3/projects/55db8128e376436dbc43a52611f93a5f/users/d562ab3d5d014f1dbf980dedad3e33c7/roles/8feebc60363c4feb870718b1721e5324 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:02:21.896348 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a461b43-0e54-48c4-8c24-269ef1c217ee None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:21.949404 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fb551d-94b4-4d41-8f19-637ae2af292d None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:21.949635 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fb551d-94b4-4d41-8f19-637ae2af292d None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:21.949883 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fb551d-94b4-4d41-8f19-637ae2af292d None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:21.951290 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 68/136] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:02:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 21:02:21.960978 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33baaad-a640-4b3e-8bf2-b274c2615f8f None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:21.962513 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33baaad-a640-4b3e-8bf2-b274c2615f8f None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:21.962513 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33baaad-a640-4b3e-8bf2-b274c2615f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:21.992928 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-b33baaad-a640-4b3e-8bf2-b274c2615f8f None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:21.996394 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-b33baaad-a640-4b3e-8bf2-b274c2615f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:22.058655 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:22.066863 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a461b43-0e54-48c4-8c24-269ef1c217ee None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:22.114804 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5850cec3-61af-4ea8-bd6a-e643ddebe473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:22.115878 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5850cec3-61af-4ea8-bd6a-e643ddebe473 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:22.116313 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5850cec3-61af-4ea8-bd6a-e643ddebe473 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:22.116734 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5850cec3-61af-4ea8-bd6a-e643ddebe473 None admin] PATH_INFO: `/v3/projects/b8f804a236884f039371233cce01ecef` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:22.118263 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5850cec3-61af-4ea8-bd6a-e643ddebe473 None admin] RBAC: Authorizing `identity:get_project(project_id=b8f804a236884f039371233cce01ecef)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:22.120334 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5850cec3-61af-4ea8-bd6a-e643ddebe473 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:22.122484 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 69/137] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Aug 4 21:02:21 2022] GET /identity/v3/projects/b8f804a236884f039371233cce01ecef => generated 298 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:02:22.129168 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5850cec3-61af-4ea8-bd6a-e643ddebe473 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:22.166172 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 69/138] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:02:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:02:22.212496 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33baaad-a640-4b3e-8bf2-b274c2615f8f None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:22.277364 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5850cec3-61af-4ea8-bd6a-e643ddebe473 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:22.322894 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3dc2e8-1fd2-49b0-8c8f-7d59f173492a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:22.323861 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3dc2e8-1fd2-49b0-8c8f-7d59f173492a None admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:22.324309 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3dc2e8-1fd2-49b0-8c8f-7d59f173492a None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:22.324732 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3dc2e8-1fd2-49b0-8c8f-7d59f173492a None admin] PATH_INFO: `/v3/projects/b8f804a236884f039371233cce01ecef/users/d562ab3d5d014f1dbf980dedad3e33c7/roles/70587326d8584316b73b380df2ae8e67` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:22.327135 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3dc2e8-1fd2-49b0-8c8f-7d59f173492a None admin] RBAC: Authorizing `identity:create_grant(project_id=b8f804a236884f039371233cce01ecef, user_id=d562ab3d5d014f1dbf980dedad3e33c7, role_id=70587326d8584316b73b380df2ae8e67)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:22.329000 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3dc2e8-1fd2-49b0-8c8f-7d59f173492a None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:22.381104 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33baaad-a640-4b3e-8bf2-b274c2615f8f None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:22.408625 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:22.417447 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 70/139] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Aug 4 21:02:22 2022] PUT /identity/v3/projects/b8f804a236884f039371233cce01ecef/users/d562ab3d5d014f1dbf980dedad3e33c7/roles/70587326d8584316b73b380df2ae8e67 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:02:22.424122 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b41eff-2287-47fe-9eae-d709faacbe76 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:22.425030 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b41eff-2287-47fe-9eae-d709faacbe76 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:22.425722 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b41eff-2287-47fe-9eae-d709faacbe76 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:22.427396 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 71/140] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:02:22 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:02:22.435714 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42130c7b-ef81-41c3-95fb-365016264182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:22.436600 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42130c7b-ef81-41c3-95fb-365016264182 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:22.436890 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42130c7b-ef81-41c3-95fb-365016264182 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:22.437171 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42130c7b-ef81-41c3-95fb-365016264182 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:22.438344 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daae826-1e53-47ad-9bf2-e3a36c20cf41 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:22.438834 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daae826-1e53-47ad-9bf2-e3a36c20cf41 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:22.439505 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daae826-1e53-47ad-9bf2-e3a36c20cf41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:22.442356 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42130c7b-ef81-41c3-95fb-365016264182 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:22.445709 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42130c7b-ef81-41c3-95fb-365016264182 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:22.445709 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 70/141] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:02:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:22.451684 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42130c7b-ef81-41c3-95fb-365016264182 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:22.471954 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-7daae826-1e53-47ad-9bf2-e3a36c20cf41 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:22.474024 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-7daae826-1e53-47ad-9bf2-e3a36c20cf41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:22.514343 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:22.616651 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 72/142] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:02:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 21:02:22.653219 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42130c7b-ef81-41c3-95fb-365016264182 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:22.658785 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daae826-1e53-47ad-9bf2-e3a36c20cf41 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:22.702753 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c19216-23cb-4ff0-9621-d1d7d57eaf69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:22.702753 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c19216-23cb-4ff0-9621-d1d7d57eaf69 None admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:22.703031 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c19216-23cb-4ff0-9621-d1d7d57eaf69 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:22.703116 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c19216-23cb-4ff0-9621-d1d7d57eaf69 None admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:22.703834 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c19216-23cb-4ff0-9621-d1d7d57eaf69 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:22.705278 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c19216-23cb-4ff0-9621-d1d7d57eaf69 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:22.801539 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 71/143] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Aug 4 21:02:22 2022] POST /identity/v3/users => generated 283 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:02:22.822459 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daae826-1e53-47ad-9bf2-e3a36c20cf41 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:22.869744 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e3c50c-6f80-4d09-9346-f29e3e6e9b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:22.870407 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e3c50c-6f80-4d09-9346-f29e3e6e9b0e None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:22.870692 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e3c50c-6f80-4d09-9346-f29e3e6e9b0e None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:22.870969 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e3c50c-6f80-4d09-9346-f29e3e6e9b0e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:22.885199 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e3c50c-6f80-4d09-9346-f29e3e6e9b0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:02:22.890257 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-a1e3c50c-6f80-4d09-9346-f29e3e6e9b0e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 21:02:22.890257 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:02:22.890257 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:02:22.890257 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:02:22.890257 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:02:22.890257 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:02:22.890257 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:02:22.890257 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:02:22.890257 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:02:22.890257 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:02:22.890257 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:02:22.890257 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:02:22.890257 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:02:22.890257 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:02:22.890257 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:02:22.890257 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:02:22.891083 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:02:22.891083 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:02:22.891083 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:02:22.891083 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:02:22.891083 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:02:22.891083 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:02:22.891083 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:02:22.891083 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:02:22.891083 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:02:22.891083 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:02:22.891083 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:02:22.891083 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:02:22.891083 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:02:22.891083 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:02:22.891083 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:02:22.891083 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:02:22.891904 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:02:22.891904 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:02:22.891904 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:02:22.891904 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:02:22.891904 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:02:22.891904 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:02:22.891904 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:02:22.891904 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:02:22.891904 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:02:22.891904 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:02:22.891904 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 21:02:22.891904 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:02:22.891904 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 73/144] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Aug 4 21:02:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:02:22.904660 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c19216-23cb-4ff0-9621-d1d7d57eaf69 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:22.978021 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:23.044327 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c19216-23cb-4ff0-9621-d1d7d57eaf69 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:23.079424 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf93f60-b9cc-446b-a1e5-70180d351eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:23.079984 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf93f60-b9cc-446b-a1e5-70180d351eaf None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:23.080268 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf93f60-b9cc-446b-a1e5-70180d351eaf None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:23.080548 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf93f60-b9cc-446b-a1e5-70180d351eaf None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:23.081272 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf93f60-b9cc-446b-a1e5-70180d351eaf None admin] RBAC: Adding query filter params (name=reader) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:23.082107 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf93f60-b9cc-446b-a1e5-70180d351eaf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:23.083589 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf93f60-b9cc-446b-a1e5-70180d351eaf None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:23.092301 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 72/145] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Aug 4 21:02:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:23.098351 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e3c50c-6f80-4d09-9346-f29e3e6e9b0e None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:23.122383 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:23.228202 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e3c50c-6f80-4d09-9346-f29e3e6e9b0e None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:23.280055 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bc9953-574b-477d-89b3-5ee902db3ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:23.280825 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bc9953-574b-477d-89b3-5ee902db3ebc None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:23.281015 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bc9953-574b-477d-89b3-5ee902db3ebc None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:23.281298 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bc9953-574b-477d-89b3-5ee902db3ebc None admin] PATH_INFO: `/v3/users/40bb688e6e224290b1de6e4873bb7143` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:23.282574 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bc9953-574b-477d-89b3-5ee902db3ebc None admin] RBAC: Authorizing `identity:get_user(user_id=40bb688e6e224290b1de6e4873bb7143)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:23.284364 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bc9953-574b-477d-89b3-5ee902db3ebc None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:23.286543 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 74/146] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Aug 4 21:02:23 2022] GET /identity/v3/users/40bb688e6e224290b1de6e4873bb7143 => generated 289 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:23.292207 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf93f60-b9cc-446b-a1e5-70180d351eaf None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:23.430460 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf93f60-b9cc-446b-a1e5-70180d351eaf None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:23.443737 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:23.477642 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f938efe-415f-45c4-b71b-a6b163e49917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:23.478639 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f938efe-415f-45c4-b71b-a6b163e49917 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:23.479115 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f938efe-415f-45c4-b71b-a6b163e49917 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:23.479624 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f938efe-415f-45c4-b71b-a6b163e49917 None admin] PATH_INFO: `/v3/projects/b8f804a236884f039371233cce01ecef` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:23.480989 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f938efe-415f-45c4-b71b-a6b163e49917 None admin] RBAC: Authorizing `identity:get_project(project_id=b8f804a236884f039371233cce01ecef)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:23.482710 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f938efe-415f-45c4-b71b-a6b163e49917 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:23.484955 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 73/147] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Aug 4 21:02:23 2022] GET /identity/v3/projects/b8f804a236884f039371233cce01ecef => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:23.492383 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bc9953-574b-477d-89b3-5ee902db3ebc None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:23.521431 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3b689f-11c4-4435-9dc6-f7e1a23fb6e7 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:23.522048 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3b689f-11c4-4435-9dc6-f7e1a23fb6e7 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:23.522518 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3b689f-11c4-4435-9dc6-f7e1a23fb6e7 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:23.524404 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 74/148] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:02:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:02:23.529700 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b606f3ae-3f35-4370-b026-2952f9a7119e None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:23.530259 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b606f3ae-3f35-4370-b026-2952f9a7119e None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:23.530724 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b606f3ae-3f35-4370-b026-2952f9a7119e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:23.551086 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-b606f3ae-3f35-4370-b026-2952f9a7119e None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:23.553569 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-b606f3ae-3f35-4370-b026-2952f9a7119e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:23.636402 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 75/149] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:02:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:02:23.648579 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:23.662238 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bc9953-574b-477d-89b3-5ee902db3ebc None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:23.674594 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b606f3ae-3f35-4370-b026-2952f9a7119e None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:23.727161 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00b125c-ef03-4361-92a6-9109848b96ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:23.727710 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00b125c-ef03-4361-92a6-9109848b96ae None admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:23.727993 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00b125c-ef03-4361-92a6-9109848b96ae None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:23.728272 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00b125c-ef03-4361-92a6-9109848b96ae None admin] PATH_INFO: `/v3/projects/b8f804a236884f039371233cce01ecef/users/40bb688e6e224290b1de6e4873bb7143/roles/5ab0887d24bc41b882f6786b7aab0fe8` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:23.732662 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00b125c-ef03-4361-92a6-9109848b96ae None admin] RBAC: Authorizing `identity:create_grant(project_id=b8f804a236884f039371233cce01ecef, user_id=40bb688e6e224290b1de6e4873bb7143, role_id=5ab0887d24bc41b882f6786b7aab0fe8)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:23.734176 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00b125c-ef03-4361-92a6-9109848b96ae None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:23.790143 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b606f3ae-3f35-4370-b026-2952f9a7119e None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:23.794003 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:23.825549 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 75/150] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Aug 4 21:02:23 2022] PUT /identity/v3/projects/b8f804a236884f039371233cce01ecef/users/40bb688e6e224290b1de6e4873bb7143/roles/5ab0887d24bc41b882f6786b7aab0fe8 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:02:23.833803 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3277344-ebbe-4fac-b1ff-0fb59a3c11f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:23.834503 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3277344-ebbe-4fac-b1ff-0fb59a3c11f0 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:23.834771 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3277344-ebbe-4fac-b1ff-0fb59a3c11f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:23.835046 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3277344-ebbe-4fac-b1ff-0fb59a3c11f0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:23.853279 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3277344-ebbe-4fac-b1ff-0fb59a3c11f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:02:23.858489 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-d3277344-ebbe-4fac-b1ff-0fb59a3c11f0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 21:02:23.858489 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:02:23.858489 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:02:23.858489 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:02:23.858489 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:02:23.858489 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:02:23.858489 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:02:23.858489 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:02:23.858489 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:02:23.858489 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:02:23.858489 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:02:23.858489 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:02:23.858489 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:02:23.858489 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:02:23.858489 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:02:23.858489 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:02:23.859411 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:02:23.859411 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:02:23.859411 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:02:23.859411 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:02:23.859411 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:02:23.859411 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:02:23.859411 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:02:23.859411 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:02:23.859411 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:02:23.859411 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:02:23.859411 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:02:23.859411 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:02:23.859411 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:02:23.859411 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:02:23.859411 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:02:23.859411 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:02:23.860205 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:02:23.860205 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:02:23.860205 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:02:23.860205 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:02:23.860205 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:02:23.860205 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:02:23.860205 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:02:23.860205 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:02:23.860205 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:02:23.860205 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:02:23.860205 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 21:02:23.860205 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:02:23.860205 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 76/151] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Aug 4 21:02:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:02:23.867876 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00b125c-ef03-4361-92a6-9109848b96ae None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:24.048362 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00b125c-ef03-4361-92a6-9109848b96ae None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:24.083608 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989445e9-3153-45e1-9b2b-b84f2bc5b520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:24.084239 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989445e9-3153-45e1-9b2b-b84f2bc5b520 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:24.084523 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989445e9-3153-45e1-9b2b-b84f2bc5b520 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:24.084798 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989445e9-3153-45e1-9b2b-b84f2bc5b520 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:24.085517 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989445e9-3153-45e1-9b2b-b84f2bc5b520 None admin] RBAC: Adding query filter params (name=member) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:24.085823 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989445e9-3153-45e1-9b2b-b84f2bc5b520 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:24.087224 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989445e9-3153-45e1-9b2b-b84f2bc5b520 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:24.096011 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 76/152] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Aug 4 21:02:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:24.102110 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3277344-ebbe-4fac-b1ff-0fb59a3c11f0 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:24.237386 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3277344-ebbe-4fac-b1ff-0fb59a3c11f0 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:24.281135 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7545440-d140-499d-bfa4-51864ae1852f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:24.281810 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7545440-d140-499d-bfa4-51864ae1852f None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:24.282093 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7545440-d140-499d-bfa4-51864ae1852f None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:24.282423 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7545440-d140-499d-bfa4-51864ae1852f None admin] PATH_INFO: `/v3/users/d562ab3d5d014f1dbf980dedad3e33c7` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:24.283972 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7545440-d140-499d-bfa4-51864ae1852f None admin] RBAC: Authorizing `identity:get_user(user_id=d562ab3d5d014f1dbf980dedad3e33c7)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:24.286047 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7545440-d140-499d-bfa4-51864ae1852f None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:24.287876 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 77/153] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Aug 4 21:02:24 2022] GET /identity/v3/users/d562ab3d5d014f1dbf980dedad3e33c7 => generated 275 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:24.293228 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989445e9-3153-45e1-9b2b-b84f2bc5b520 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:24.305875 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:24.556645 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989445e9-3153-45e1-9b2b-b84f2bc5b520 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:24.617483 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c62ba89-51fb-4b3a-a98b-8773c3e5d719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:24.618125 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c62ba89-51fb-4b3a-a98b-8773c3e5d719 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:24.618409 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c62ba89-51fb-4b3a-a98b-8773c3e5d719 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:24.618682 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c62ba89-51fb-4b3a-a98b-8773c3e5d719 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:24.750382 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c62ba89-51fb-4b3a-a98b-8773c3e5d719 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:24.752780 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c62ba89-51fb-4b3a-a98b-8773c3e5d719 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:24.756802 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:24.910104 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 77/154] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Aug 4 21:02:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 617 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 21:02:24.918513 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7545440-d140-499d-bfa4-51864ae1852f None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:24.999705 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:25.093062 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7545440-d140-499d-bfa4-51864ae1852f None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:25.136466 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c03175b-53b4-49f4-b509-f23077cd2cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:25.137123 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c03175b-53b4-49f4-b509-f23077cd2cb8 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:25.137776 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c03175b-53b4-49f4-b509-f23077cd2cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:25.138053 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c03175b-53b4-49f4-b509-f23077cd2cb8 None admin] PATH_INFO: `/v3/projects/b8f804a236884f039371233cce01ecef` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:25.139286 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c03175b-53b4-49f4-b509-f23077cd2cb8 None admin] RBAC: Authorizing `identity:get_project(project_id=b8f804a236884f039371233cce01ecef)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:25.140768 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c03175b-53b4-49f4-b509-f23077cd2cb8 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:25.142433 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 78/155] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Aug 4 21:02:24 2022] GET /identity/v3/projects/b8f804a236884f039371233cce01ecef => generated 298 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:25.149016 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c62ba89-51fb-4b3a-a98b-8773c3e5d719 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:25.152644 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d9972-6243-4816-b492-3e25860ba745 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:25.152930 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d9972-6243-4816-b492-3e25860ba745 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:25.153199 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d9972-6243-4816-b492-3e25860ba745 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:25.155180 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 79/156] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:02:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 21:02:25.163409 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce3f7d8-cd96-4487-8bed-1f47a7c50dcb None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:25.163409 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce3f7d8-cd96-4487-8bed-1f47a7c50dcb None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:25.163409 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce3f7d8-cd96-4487-8bed-1f47a7c50dcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:25.188436 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-6ce3f7d8-cd96-4487-8bed-1f47a7c50dcb None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:25.189628 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-6ce3f7d8-cd96-4487-8bed-1f47a7c50dcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:25.264153 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 80/157] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:02:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 21:02:25.286841 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce3f7d8-cd96-4487-8bed-1f47a7c50dcb None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:25.318439 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:25.356843 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c62ba89-51fb-4b3a-a98b-8773c3e5d719 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:25.369420 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:25.397804 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce3f7d8-cd96-4487-8bed-1f47a7c50dcb None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:25.401231 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7606038-772b-4b52-9a9b-1fb342a8e883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:25.402118 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7606038-772b-4b52-9a9b-1fb342a8e883 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:25.402605 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7606038-772b-4b52-9a9b-1fb342a8e883 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:25.403082 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7606038-772b-4b52-9a9b-1fb342a8e883 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:25.404155 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7606038-772b-4b52-9a9b-1fb342a8e883 None admin] RBAC: Adding query filter params (role.id=8feebc60363c4feb870718b1721e5324, scope.project.id=b8f804a236884f039371233cce01ecef, user.id=d562ab3d5d014f1dbf980dedad3e33c7) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:25.404683 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7606038-772b-4b52-9a9b-1fb342a8e883 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:25.406375 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7606038-772b-4b52-9a9b-1fb342a8e883 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:25.418782 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 78/158] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Aug 4 21:02:25 2022] GET /identity/v3/role_assignments?user.id=d562ab3d5d014f1dbf980dedad3e33c7&scope.project.id=b8f804a236884f039371233cce01ecef&role.id=8feebc60363c4feb870718b1721e5324 => generated 608 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:25.455274 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b90ead-ffa4-4f49-8d96-a7acbebd9449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:25.455957 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b90ead-ffa4-4f49-8d96-a7acbebd9449 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:25.456232 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b90ead-ffa4-4f49-8d96-a7acbebd9449 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:25.456535 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b90ead-ffa4-4f49-8d96-a7acbebd9449 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:25.476545 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b90ead-ffa4-4f49-8d96-a7acbebd9449 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:02:25.481825 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-83b90ead-ffa4-4f49-8d96-a7acbebd9449 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:02:25.481825 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:02:25.481825 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:02:25.481825 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:02:25.481825 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:02:25.481825 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:02:25.481825 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:02:25.481825 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:02:25.481825 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:02:25.481825 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:02:25.481825 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:02:25.481825 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:02:25.481825 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:02:25.481825 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:02:25.481825 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:02:25.481825 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:02:25.482723 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:02:25.482723 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:02:25.482723 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:02:25.482723 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:02:25.482723 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:02:25.482723 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:02:25.482723 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:02:25.482723 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:02:25.482723 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:02:25.482723 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:02:25.482723 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:02:25.482723 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:02:25.482723 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:02:25.482723 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:02:25.482723 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:02:25.482723 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:02:25.483561 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:02:25.483561 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:02:25.483561 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:02:25.483561 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:02:25.483561 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:02:25.483561 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:02:25.483561 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:02:25.483561 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:02:25.483561 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:02:25.483561 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:02:25.483561 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:02:25.483561 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:02:25.487433 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 81/159] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Aug 4 21:02:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:02:25.492425 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7606038-772b-4b52-9a9b-1fb342a8e883 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:25.654870 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7606038-772b-4b52-9a9b-1fb342a8e883 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:25.661985 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bb346c-3a8d-4c41-a780-e23957909b52 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:25.662274 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bb346c-3a8d-4c41-a780-e23957909b52 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:25.662556 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bb346c-3a8d-4c41-a780-e23957909b52 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:25.664089 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 82/160] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:02:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 21:02:25.681404 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c80e27b-9107-472e-b36b-6636f3335981 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:25.681690 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c80e27b-9107-472e-b36b-6636f3335981 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:25.681883 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c80e27b-9107-472e-b36b-6636f3335981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:25.701156 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40956f7f-e534-4c02-a238-163723e4de59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:25.701801 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40956f7f-e534-4c02-a238-163723e4de59 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:25.702085 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40956f7f-e534-4c02-a238-163723e4de59 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:25.702360 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40956f7f-e534-4c02-a238-163723e4de59 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:25.703146 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40956f7f-e534-4c02-a238-163723e4de59 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:25.703489 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40956f7f-e534-4c02-a238-163723e4de59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:25.704913 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40956f7f-e534-4c02-a238-163723e4de59 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:25.715485 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-4c80e27b-9107-472e-b36b-6636f3335981 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:25.716705 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-4c80e27b-9107-472e-b36b-6636f3335981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:25.721475 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 79/161] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Aug 4 21:02:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:25.728639 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40956f7f-e534-4c02-a238-163723e4de59 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:25.816933 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:25.897838 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 83/162] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:02:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 21:02:25.938034 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c80e27b-9107-472e-b36b-6636f3335981 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:25.948774 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40956f7f-e534-4c02-a238-163723e4de59 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:25.985783 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:26.005471 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71307224-dbdf-4dac-9955-8f69544fd1a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:26.006138 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71307224-dbdf-4dac-9955-8f69544fd1a9 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:26.006426 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71307224-dbdf-4dac-9955-8f69544fd1a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:26.006703 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71307224-dbdf-4dac-9955-8f69544fd1a9 None admin] PATH_INFO: `/v3/users/bc6bad33dfb24427b1ccf30de722b45a` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:26.007922 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71307224-dbdf-4dac-9955-8f69544fd1a9 None admin] RBAC: Authorizing `identity:get_user(user_id=bc6bad33dfb24427b1ccf30de722b45a)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:26.011739 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71307224-dbdf-4dac-9955-8f69544fd1a9 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:26.013435 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 80/163] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Aug 4 21:02:25 2022] GET /identity/v3/users/bc6bad33dfb24427b1ccf30de722b45a => generated 247 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:02:26.019336 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71307224-dbdf-4dac-9955-8f69544fd1a9 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:26.168849 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c80e27b-9107-472e-b36b-6636f3335981 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:26.211243 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7ec6f1-10ed-4d71-9908-31404acf8c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:26.211884 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7ec6f1-10ed-4d71-9908-31404acf8c7e None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:26.212183 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7ec6f1-10ed-4d71-9908-31404acf8c7e None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:26.212457 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7ec6f1-10ed-4d71-9908-31404acf8c7e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:26.224124 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7ec6f1-10ed-4d71-9908-31404acf8c7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:02:26.232748 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-ef7ec6f1-10ed-4d71-9908-31404acf8c7e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 21:02:26.232748 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:02:26.232748 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:02:26.232748 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:02:26.232748 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:02:26.232748 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:02:26.232748 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:02:26.232748 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:02:26.232748 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:02:26.232748 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:02:26.232748 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:02:26.232748 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:02:26.232748 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:02:26.232748 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:02:26.232748 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:02:26.232748 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:02:26.233841 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:02:26.233841 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:02:26.233841 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:02:26.233841 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:02:26.233841 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:02:26.233841 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:02:26.233841 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:02:26.233841 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:02:26.233841 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:02:26.233841 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:02:26.233841 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:02:26.233841 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:02:26.233841 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:02:26.233841 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:02:26.233841 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:02:26.233841 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:02:26.234723 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:02:26.234723 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:02:26.234723 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:02:26.234723 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:02:26.234723 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:02:26.234723 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:02:26.234723 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:02:26.234723 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:02:26.234723 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:02:26.234723 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:02:26.234723 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 21:02:26.234723 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:02:26.234723 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 84/164] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Aug 4 21:02:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:02:26.234723 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b1d060-0969-419c-ba5e-5da127e75946 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:26.235479 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b1d060-0969-419c-ba5e-5da127e75946 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:26.235479 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b1d060-0969-419c-ba5e-5da127e75946 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:26.236153 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 85/165] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:02:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:02:26.242370 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b1d060-0969-419c-ba5e-5da127e75946 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:26.265366 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:26.270126 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71307224-dbdf-4dac-9955-8f69544fd1a9 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:26.306889 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235f0502-bf65-488a-a372-851f55242dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:26.308348 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235f0502-bf65-488a-a372-851f55242dde None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:26.308861 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235f0502-bf65-488a-a372-851f55242dde None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:26.309444 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235f0502-bf65-488a-a372-851f55242dde None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:26.354435 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:26.383265 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b1d060-0969-419c-ba5e-5da127e75946 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:26.386549 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235f0502-bf65-488a-a372-851f55242dde None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:26.389237 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235f0502-bf65-488a-a372-851f55242dde None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:26.432196 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d37c059-860e-45d0-9d86-e94610692226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:26.432860 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d37c059-860e-45d0-9d86-e94610692226 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:26.433159 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d37c059-860e-45d0-9d86-e94610692226 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:26.433431 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d37c059-860e-45d0-9d86-e94610692226 None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:26.434712 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d37c059-860e-45d0-9d86-e94610692226 None admin] RBAC: Adding query filter params (name=member) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:26.435018 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d37c059-860e-45d0-9d86-e94610692226 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:26.436618 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d37c059-860e-45d0-9d86-e94610692226 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:26.448723 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 86/166] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Aug 4 21:02:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:02:26.450196 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a34b8e0-6f6f-424a-a5d0-00b27272d809 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:26.450539 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a34b8e0-6f6f-424a-a5d0-00b27272d809 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:26.450801 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a34b8e0-6f6f-424a-a5d0-00b27272d809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:26.475299 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-0a34b8e0-6f6f-424a-a5d0-00b27272d809 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:26.476677 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-0a34b8e0-6f6f-424a-a5d0-00b27272d809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:26.506705 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:26.511571 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 81/167] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Aug 4 21:02:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 04 21:02:26.512935 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235f0502-bf65-488a-a372-851f55242dde None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:26.571347 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 87/168] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:02:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:02:26.572319 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a34b8e0-6f6f-424a-a5d0-00b27272d809 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:26.660061 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:26.700612 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a34b8e0-6f6f-424a-a5d0-00b27272d809 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:26.744752 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92e4a67-d01d-4c44-b300-2e76a4efa454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:26.745914 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92e4a67-d01d-4c44-b300-2e76a4efa454 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:26.746335 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92e4a67-d01d-4c44-b300-2e76a4efa454 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:26.746768 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92e4a67-d01d-4c44-b300-2e76a4efa454 None admin] PATH_INFO: `/v3/projects/b8f804a236884f039371233cce01ecef` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:26.748029 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235f0502-bf65-488a-a372-851f55242dde None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:26.748360 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92e4a67-d01d-4c44-b300-2e76a4efa454 None admin] RBAC: Authorizing `identity:get_project(project_id=b8f804a236884f039371233cce01ecef)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:26.750244 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92e4a67-d01d-4c44-b300-2e76a4efa454 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:26.752394 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 88/169] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Aug 4 21:02:26 2022] GET /identity/v3/projects/b8f804a236884f039371233cce01ecef => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:26.753365 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92e4a67-d01d-4c44-b300-2e76a4efa454 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:26.838459 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bf9382-84e0-41de-b07f-cb28bfacb685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:26.839066 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bf9382-84e0-41de-b07f-cb28bfacb685 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:26.839476 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bf9382-84e0-41de-b07f-cb28bfacb685 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:26.839848 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bf9382-84e0-41de-b07f-cb28bfacb685 None admin] PATH_INFO: `/v3/users/d562ab3d5d014f1dbf980dedad3e33c7` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:26.841171 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bf9382-84e0-41de-b07f-cb28bfacb685 None admin] RBAC: Authorizing `identity:get_user(user_id=d562ab3d5d014f1dbf980dedad3e33c7)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:26.843567 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bf9382-84e0-41de-b07f-cb28bfacb685 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:26.850629 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 82/170] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Aug 4 21:02:26 2022] GET /identity/v3/users/d562ab3d5d014f1dbf980dedad3e33c7 => generated 275 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:26.851624 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bf9382-84e0-41de-b07f-cb28bfacb685 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:26.879194 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92e4a67-d01d-4c44-b300-2e76a4efa454 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:26.921361 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3575a44e-de13-4e6f-b2af-a2e8701e10ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:26.922243 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3575a44e-de13-4e6f-b2af-a2e8701e10ce None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:26.922662 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3575a44e-de13-4e6f-b2af-a2e8701e10ce None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:26.923340 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3575a44e-de13-4e6f-b2af-a2e8701e10ce None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:26.933474 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:26.939495 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3575a44e-de13-4e6f-b2af-a2e8701e10ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:02:26.946439 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-3575a44e-de13-4e6f-b2af-a2e8701e10ce None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 21:02:26.946439 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:02:26.946439 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:02:26.946439 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:02:26.946439 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:02:26.946439 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:02:26.946439 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:02:26.946439 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:02:26.946439 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:02:26.946439 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:02:26.946439 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:02:26.946439 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:02:26.946439 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:02:26.946439 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:02:26.946439 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:02:26.946439 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:02:26.948946 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:02:26.948946 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:02:26.948946 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:02:26.948946 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:02:26.948946 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:02:26.948946 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:02:26.948946 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:02:26.948946 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:02:26.948946 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:02:26.948946 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:02:26.948946 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:02:26.948946 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:02:26.948946 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:02:26.948946 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:02:26.948946 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:02:26.948946 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:02:26.949736 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:02:26.949736 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:02:26.949736 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:02:26.949736 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:02:26.949736 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:02:26.949736 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:02:26.949736 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:02:26.949736 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:02:26.949736 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:02:26.949736 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:02:26.949736 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 21:02:26.949736 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:02:26.950963 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 89/171] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Aug 4 21:02:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:02:26.952490 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3575a44e-de13-4e6f-b2af-a2e8701e10ce None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:26.998514 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:27.083561 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3575a44e-de13-4e6f-b2af-a2e8701e10ce None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:27.086091 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bf9382-84e0-41de-b07f-cb28bfacb685 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:27.125454 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c3d67f-6570-40ff-abc9-323d558eb023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:27.126414 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c3d67f-6570-40ff-abc9-323d558eb023 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:27.126879 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c3d67f-6570-40ff-abc9-323d558eb023 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:27.127368 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c3d67f-6570-40ff-abc9-323d558eb023 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:27.156973 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4d1b52-797c-4841-9269-a4f5fff56053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:27.157610 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d1b52-797c-4841-9269-a4f5fff56053 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:27.157899 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d1b52-797c-4841-9269-a4f5fff56053 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:27.158445 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d1b52-797c-4841-9269-a4f5fff56053 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:27.159378 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4d1b52-797c-4841-9269-a4f5fff56053 None admin] RBAC: Adding query filter params (role.id=a432dd4735944ecfab2fa5ce1705d356, scope.project.id=b8f804a236884f039371233cce01ecef, user.id=bc6bad33dfb24427b1ccf30de722b45a) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:27.160233 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4d1b52-797c-4841-9269-a4f5fff56053 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:27.161698 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4d1b52-797c-4841-9269-a4f5fff56053 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:27.179049 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 83/172] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Aug 4 21:02:26 2022] GET /identity/v3/role_assignments?user.id=bc6bad33dfb24427b1ccf30de722b45a&scope.project.id=b8f804a236884f039371233cce01ecef&role.id=a432dd4735944ecfab2fa5ce1705d356 => generated 608 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:27.179049 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4d1b52-797c-4841-9269-a4f5fff56053 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:27.219857 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:27.229737 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c3d67f-6570-40ff-abc9-323d558eb023 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:27.235339 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c3d67f-6570-40ff-abc9-323d558eb023 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:27.339983 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4d1b52-797c-4841-9269-a4f5fff56053 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:27.387408 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:27.389359 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f8d5bf-a43c-4260-8c18-4feae8cae16a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:27.390236 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f8d5bf-a43c-4260-8c18-4feae8cae16a None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:27.390520 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f8d5bf-a43c-4260-8c18-4feae8cae16a None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:27.390796 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f8d5bf-a43c-4260-8c18-4feae8cae16a None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:27.391570 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f8d5bf-a43c-4260-8c18-4feae8cae16a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:27.391878 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f8d5bf-a43c-4260-8c18-4feae8cae16a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:27.393330 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f8d5bf-a43c-4260-8c18-4feae8cae16a None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:27.409359 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 90/173] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Aug 4 21:02:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 21:02:27.418163 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 84/174] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Aug 4 21:02:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:27.423993 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c3d67f-6570-40ff-abc9-323d558eb023 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:27.427375 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f8d5bf-a43c-4260-8c18-4feae8cae16a None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:27.564601 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f8d5bf-a43c-4260-8c18-4feae8cae16a None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:27.571714 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c3d67f-6570-40ff-abc9-323d558eb023 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:27.573256 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:27.609072 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8dc7ba-d2d8-4fd2-8a09-c6f5b3484399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:27.609654 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8dc7ba-d2d8-4fd2-8a09-c6f5b3484399 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:27.609943 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8dc7ba-d2d8-4fd2-8a09-c6f5b3484399 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:27.610223 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8dc7ba-d2d8-4fd2-8a09-c6f5b3484399 None admin] PATH_INFO: `/v3/users/d562ab3d5d014f1dbf980dedad3e33c7` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:27.611472 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8dc7ba-d2d8-4fd2-8a09-c6f5b3484399 None admin] RBAC: Authorizing `identity:get_user(user_id=d562ab3d5d014f1dbf980dedad3e33c7)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:27.612039 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5701d4f-383d-4872-9cc7-25cfe3d1fc12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:27.613243 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5701d4f-383d-4872-9cc7-25cfe3d1fc12 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:27.613482 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8dc7ba-d2d8-4fd2-8a09-c6f5b3484399 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:27.613588 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5701d4f-383d-4872-9cc7-25cfe3d1fc12 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:27.613785 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5701d4f-383d-4872-9cc7-25cfe3d1fc12 None admin] PATH_INFO: `/v3/projects/55db8128e376436dbc43a52611f93a5f` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:27.615410 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 91/175] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Aug 4 21:02:27 2022] GET /identity/v3/users/d562ab3d5d014f1dbf980dedad3e33c7 => generated 275 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:27.616108 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5701d4f-383d-4872-9cc7-25cfe3d1fc12 None admin] RBAC: Authorizing `identity:get_project(project_id=55db8128e376436dbc43a52611f93a5f)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:27.616413 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d40c4cc-6f5d-4b92-89a4-5984e78c234f None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:27.616702 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d40c4cc-6f5d-4b92-89a4-5984e78c234f None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:27.616982 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d40c4cc-6f5d-4b92-89a4-5984e78c234f None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:27.617675 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5701d4f-383d-4872-9cc7-25cfe3d1fc12 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:27.618351 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 92/176] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:02:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:02:27.620561 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 85/177] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Aug 4 21:02:27 2022] GET /identity/v3/projects/55db8128e376436dbc43a52611f93a5f => generated 312 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:27.623348 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5701d4f-383d-4872-9cc7-25cfe3d1fc12 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:27.623813 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598709ca-edc2-4cde-ae58-92d19add6afe None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:27.624111 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598709ca-edc2-4cde-ae58-92d19add6afe None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:27.624387 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598709ca-edc2-4cde-ae58-92d19add6afe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:27.643494 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-598709ca-edc2-4cde-ae58-92d19add6afe None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:27.645271 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-598709ca-edc2-4cde-ae58-92d19add6afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:27.722154 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 93/178] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:02:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 21:02:27.722920 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598709ca-edc2-4cde-ae58-92d19add6afe None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:27.728544 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:27.792906 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5701d4f-383d-4872-9cc7-25cfe3d1fc12 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:27.852353 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598709ca-edc2-4cde-ae58-92d19add6afe None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:27.854900 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:27.873713 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ea04da-91af-4f68-a5e1-385a3b06b241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:27.873713 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea04da-91af-4f68-a5e1-385a3b06b241 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:27.873713 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea04da-91af-4f68-a5e1-385a3b06b241 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:27.874039 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea04da-91af-4f68-a5e1-385a3b06b241 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:27.888611 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9b2f1b-e583-4ef6-a4c0-0a2facf1984f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:27.889250 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9b2f1b-e583-4ef6-a4c0-0a2facf1984f None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:27.889534 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9b2f1b-e583-4ef6-a4c0-0a2facf1984f None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:27.889808 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9b2f1b-e583-4ef6-a4c0-0a2facf1984f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:27.890622 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9b2f1b-e583-4ef6-a4c0-0a2facf1984f None admin] RBAC: Adding query filter params (role.id=8feebc60363c4feb870718b1721e5324, scope.project.id=55db8128e376436dbc43a52611f93a5f, user.id=d562ab3d5d014f1dbf980dedad3e33c7) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:27.890936 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9b2f1b-e583-4ef6-a4c0-0a2facf1984f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:27.892723 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9b2f1b-e583-4ef6-a4c0-0a2facf1984f None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:27.899942 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 94/179] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Aug 4 21:02:27 2022] GET /identity/v3/role_assignments?user.id=d562ab3d5d014f1dbf980dedad3e33c7&scope.project.id=55db8128e376436dbc43a52611f93a5f&role.id=8feebc60363c4feb870718b1721e5324 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:27.900863 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9b2f1b-e583-4ef6-a4c0-0a2facf1984f None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:27.996342 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ea04da-91af-4f68-a5e1-385a3b06b241 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:27.998133 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ea04da-91af-4f68-a5e1-385a3b06b241 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:28.098305 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:28.102483 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9b2f1b-e583-4ef6-a4c0-0a2facf1984f None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:28.143559 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 86/180] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Aug 4 21:02:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 21:02:28.145473 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89978369-1f2a-43ca-89f7-5d4c3b3b22cc None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:28.145660 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89978369-1f2a-43ca-89f7-5d4c3b3b22cc None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:28.145938 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89978369-1f2a-43ca-89f7-5d4c3b3b22cc None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:28.147410 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 87/181] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:02:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:02:28.148299 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee76ae82-73b1-43ca-975d-854334c114a7 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:28.149703 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee76ae82-73b1-43ca-975d-854334c114a7 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:28.149703 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee76ae82-73b1-43ca-975d-854334c114a7 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:28.150018 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 88/182] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:02:28 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:02:28.151035 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59c61d5-5cee-4854-a8d1-8fd28c5de398 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:28.151332 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59c61d5-5cee-4854-a8d1-8fd28c5de398 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:28.151606 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59c61d5-5cee-4854-a8d1-8fd28c5de398 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:28.153626 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 89/183] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:02:28 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:02:28.153626 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59c61d5-5cee-4854-a8d1-8fd28c5de398 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:28.156896 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5847d4-9987-4c60-b7e8-32c71ba34db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:28.157711 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5847d4-9987-4c60-b7e8-32c71ba34db7 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:28.158002 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5847d4-9987-4c60-b7e8-32c71ba34db7 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:28.158542 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5847d4-9987-4c60-b7e8-32c71ba34db7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:28.174258 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5847d4-9987-4c60-b7e8-32c71ba34db7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:02:28.179827 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-bc5847d4-9987-4c60-b7e8-32c71ba34db7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:02:28.179827 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:02:28.179827 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:02:28.179827 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:02:28.179827 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:02:28.179827 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:02:28.179827 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:02:28.179827 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:02:28.179827 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:02:28.179827 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:02:28.179827 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:02:28.179827 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:02:28.179827 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:02:28.179827 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:02:28.179827 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:02:28.179827 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:02:28.180881 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:02:28.180881 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:02:28.180881 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:02:28.180881 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:02:28.180881 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:02:28.180881 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:02:28.180881 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:02:28.180881 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:02:28.180881 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:02:28.180881 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:02:28.180881 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:02:28.180881 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:02:28.180881 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:02:28.180881 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:02:28.180881 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:02:28.180881 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:02:28.181840 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:02:28.181840 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:02:28.181840 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:02:28.181840 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:02:28.181840 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:02:28.181840 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:02:28.181840 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:02:28.181840 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:02:28.181840 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:02:28.181840 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:02:28.181840 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:02:28.181840 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:02:28.181840 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 95/184] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Aug 4 21:02:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:02:28.182599 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bacda89-6617-4096-98fd-7bfb3f3ee1dc None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:28.182599 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bacda89-6617-4096-98fd-7bfb3f3ee1dc None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:28.182720 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bacda89-6617-4096-98fd-7bfb3f3ee1dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:28.203495 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-6bacda89-6617-4096-98fd-7bfb3f3ee1dc None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:28.204829 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-6bacda89-6617-4096-98fd-7bfb3f3ee1dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:28.223454 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:28.268670 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59c61d5-5cee-4854-a8d1-8fd28c5de398 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:28.273597 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 96/185] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:02:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:02:28.274975 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0abd42-53b1-4dde-84c1-412684cb7492 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:28.275268 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0abd42-53b1-4dde-84c1-412684cb7492 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:28.275572 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0abd42-53b1-4dde-84c1-412684cb7492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:28.293298 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-6a0abd42-53b1-4dde-84c1-412684cb7492 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:28.294472 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-6a0abd42-53b1-4dde-84c1-412684cb7492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:28.306948 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3f6290-9c36-4c1f-9449-b440a79e7392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:28.307620 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3f6290-9c36-4c1f-9449-b440a79e7392 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:28.307903 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3f6290-9c36-4c1f-9449-b440a79e7392 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:28.308176 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3f6290-9c36-4c1f-9449-b440a79e7392 None admin] PATH_INFO: `/v3/projects/b8f804a236884f039371233cce01ecef` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:28.309302 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3f6290-9c36-4c1f-9449-b440a79e7392 None admin] RBAC: Authorizing `identity:get_project(project_id=b8f804a236884f039371233cce01ecef)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:28.310748 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3f6290-9c36-4c1f-9449-b440a79e7392 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:28.312570 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 90/186] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Aug 4 21:02:28 2022] GET /identity/v3/projects/b8f804a236884f039371233cce01ecef => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:28.313518 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1891c3-2789-4e19-94d4-cd130c5eb402 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:28.313798 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1891c3-2789-4e19-94d4-cd130c5eb402 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:28.314070 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1891c3-2789-4e19-94d4-cd130c5eb402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:28.331786 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-db1891c3-2789-4e19-94d4-cd130c5eb402 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:28.333009 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-db1891c3-2789-4e19-94d4-cd130c5eb402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:28.355155 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:28.362985 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 97/187] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:02:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:02:28.383473 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0abd42-53b1-4dde-84c1-412684cb7492 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:28.401816 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 91/188] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:02:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:02:28.403212 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1891c3-2789-4e19-94d4-cd130c5eb402 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:28.447272 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:28.469060 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0abd42-53b1-4dde-84c1-412684cb7492 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:28.501928 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1891c3-2789-4e19-94d4-cd130c5eb402 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:28.508176 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807234a6-7891-41f4-8ba0-f0ad771df50c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:28.508797 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807234a6-7891-41f4-8ba0-f0ad771df50c None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:28.509076 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807234a6-7891-41f4-8ba0-f0ad771df50c None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:28.509347 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807234a6-7891-41f4-8ba0-f0ad771df50c None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:28.510053 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807234a6-7891-41f4-8ba0-f0ad771df50c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:28.510352 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807234a6-7891-41f4-8ba0-f0ad771df50c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:28.511758 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807234a6-7891-41f4-8ba0-f0ad771df50c None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:28.519691 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 98/189] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Aug 4 21:02:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:28.520455 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807234a6-7891-41f4-8ba0-f0ad771df50c None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:28.536331 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c0700a-654f-448a-8f15-297cbcc9dadf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:28.536981 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c0700a-654f-448a-8f15-297cbcc9dadf None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:28.537266 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c0700a-654f-448a-8f15-297cbcc9dadf None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:28.537536 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c0700a-654f-448a-8f15-297cbcc9dadf None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:28.547677 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c0700a-654f-448a-8f15-297cbcc9dadf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:02:28.553257 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-c7c0700a-654f-448a-8f15-297cbcc9dadf None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 21:02:28.553257 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:02:28.553257 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:02:28.553257 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:02:28.553257 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:02:28.553257 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:02:28.553257 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:02:28.553257 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:02:28.553257 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:02:28.553257 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:02:28.553257 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:02:28.553257 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:02:28.553257 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:02:28.553257 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:02:28.553257 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:02:28.553257 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:02:28.554127 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:02:28.554127 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:02:28.554127 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:02:28.554127 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:02:28.554127 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:02:28.554127 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:02:28.554127 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:02:28.554127 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:02:28.554127 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:02:28.554127 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:02:28.554127 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:02:28.554127 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:02:28.554127 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:02:28.554127 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:02:28.554127 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:02:28.554127 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:02:28.554992 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:02:28.554992 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:02:28.554992 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:02:28.554992 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:02:28.554992 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:02:28.554992 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:02:28.554992 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:02:28.554992 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:02:28.554992 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:02:28.554992 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:02:28.554992 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 21:02:28.554992 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:02:28.554992 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 92/190] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Aug 4 21:02:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:02:28.555708 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c0700a-654f-448a-8f15-297cbcc9dadf None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:28.576024 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:28.620606 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807234a6-7891-41f4-8ba0-f0ad771df50c None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:28.653652 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b43587-fe9e-4333-ad6b-f53ae4fca4af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:28.653826 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c0700a-654f-448a-8f15-297cbcc9dadf None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:28.654199 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b43587-fe9e-4333-ad6b-f53ae4fca4af None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:28.654477 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b43587-fe9e-4333-ad6b-f53ae4fca4af None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:28.654749 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b43587-fe9e-4333-ad6b-f53ae4fca4af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:28.655565 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b43587-fe9e-4333-ad6b-f53ae4fca4af None admin] RBAC: Adding query filter params (role.id=70587326d8584316b73b380df2ae8e67, scope.project.id=b8f804a236884f039371233cce01ecef, user.id=d562ab3d5d014f1dbf980dedad3e33c7) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:28.655880 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b43587-fe9e-4333-ad6b-f53ae4fca4af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:28.657324 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b43587-fe9e-4333-ad6b-f53ae4fca4af None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:28.663719 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 99/191] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Aug 4 21:02:28 2022] GET /identity/v3/role_assignments?user.id=d562ab3d5d014f1dbf980dedad3e33c7&scope.project.id=b8f804a236884f039371233cce01ecef&role.id=70587326d8584316b73b380df2ae8e67 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:28.664476 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b43587-fe9e-4333-ad6b-f53ae4fca4af None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:28.685701 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:28.692835 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc18547-fc09-475c-8687-c774206fc51d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:28.693555 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc18547-fc09-475c-8687-c774206fc51d None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:28.693946 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc18547-fc09-475c-8687-c774206fc51d None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:28.700083 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc18547-fc09-475c-8687-c774206fc51d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:28.700083 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc18547-fc09-475c-8687-c774206fc51d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:28.700083 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc18547-fc09-475c-8687-c774206fc51d None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:28.700083 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 93/192] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:02:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:28.700083 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc18547-fc09-475c-8687-c774206fc51d None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:28.790250 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b43587-fe9e-4333-ad6b-f53ae4fca4af None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:28.801331 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:28.818185 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc18547-fc09-475c-8687-c774206fc51d None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:28.830402 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f0d40c-c415-4c49-b3b1-57ccc766c19a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:28.831474 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f0d40c-c415-4c49-b3b1-57ccc766c19a None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:28.831474 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f0d40c-c415-4c49-b3b1-57ccc766c19a None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:28.831659 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f0d40c-c415-4c49-b3b1-57ccc766c19a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:28.846492 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f0d40c-c415-4c49-b3b1-57ccc766c19a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:02:28.852552 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-c3f0d40c-c415-4c49-b3b1-57ccc766c19a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 21:02:28.852552 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:02:28.852552 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:02:28.852552 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:02:28.852552 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:02:28.852552 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:02:28.852552 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:02:28.852552 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:02:28.852552 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:02:28.852552 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:02:28.852552 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:02:28.852552 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:02:28.852552 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:02:28.852552 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:02:28.852552 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:02:28.852552 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:02:28.853449 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:02:28.853449 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:02:28.853449 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:02:28.853449 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:02:28.853449 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:02:28.853449 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:02:28.853449 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:02:28.853449 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:02:28.853449 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:02:28.853449 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:02:28.853449 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:02:28.853449 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:02:28.853449 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:02:28.853449 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:02:28.853449 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:02:28.853449 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:02:28.854238 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:02:28.854238 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:02:28.854238 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:02:28.854238 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:02:28.854238 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:02:28.854238 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:02:28.854238 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:02:28.854238 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:02:28.854238 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:02:28.854238 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:02:28.854238 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 21:02:28.854238 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:02:28.856934 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 100/193] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Aug 4 21:02:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:02:28.856934 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f0d40c-c415-4c49-b3b1-57ccc766c19a None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:28.866855 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d132315-e152-49b3-ba71-17403f790e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:28.867028 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d132315-e152-49b3-ba71-17403f790e3c None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:28.867307 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d132315-e152-49b3-ba71-17403f790e3c None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:28.867618 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d132315-e152-49b3-ba71-17403f790e3c None admin] PATH_INFO: `/v3/users/93d8deefa5254382b257e6e3fbe3d481` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:28.885548 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d132315-e152-49b3-ba71-17403f790e3c None admin] RBAC: Authorizing `identity:get_user(user_id=93d8deefa5254382b257e6e3fbe3d481)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:28.885548 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d132315-e152-49b3-ba71-17403f790e3c None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:28.887176 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 94/194] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Aug 4 21:02:28 2022] GET /identity/v3/users/93d8deefa5254382b257e6e3fbe3d481 => generated 283 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:28.888484 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d132315-e152-49b3-ba71-17403f790e3c None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:28.936003 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:28.968881 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f0d40c-c415-4c49-b3b1-57ccc766c19a None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:28.987919 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d132315-e152-49b3-ba71-17403f790e3c None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:29.004913 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb62df8-8640-4238-8d05-d2161a13e8de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:29.005531 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb62df8-8640-4238-8d05-d2161a13e8de None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:29.005812 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb62df8-8640-4238-8d05-d2161a13e8de None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:29.006082 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb62df8-8640-4238-8d05-d2161a13e8de None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:29.006819 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb62df8-8640-4238-8d05-d2161a13e8de None admin] RBAC: Adding query filter params (name=reader) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:29.007126 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb62df8-8640-4238-8d05-d2161a13e8de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:29.008540 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb62df8-8640-4238-8d05-d2161a13e8de None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:29.016642 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 101/195] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Aug 4 21:02:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:29.017514 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb62df8-8640-4238-8d05-d2161a13e8de None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:29.022251 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908a492c-3e61-4cd1-81f8-494ff413c8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:29.022876 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908a492c-3e61-4cd1-81f8-494ff413c8ee None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:29.023157 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908a492c-3e61-4cd1-81f8-494ff413c8ee None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:29.023476 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908a492c-3e61-4cd1-81f8-494ff413c8ee None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:29.024165 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908a492c-3e61-4cd1-81f8-494ff413c8ee None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:29.025628 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908a492c-3e61-4cd1-81f8-494ff413c8ee None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:29.114723 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb62df8-8640-4238-8d05-d2161a13e8de None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:29.141913 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:29.147394 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633cc316-e4b6-4e69-8a5f-d31ae13d274f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:29.148012 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633cc316-e4b6-4e69-8a5f-d31ae13d274f None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:29.148297 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633cc316-e4b6-4e69-8a5f-d31ae13d274f None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:29.148574 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633cc316-e4b6-4e69-8a5f-d31ae13d274f None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:29.149294 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633cc316-e4b6-4e69-8a5f-d31ae13d274f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:29.149601 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633cc316-e4b6-4e69-8a5f-d31ae13d274f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:29.150175 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:29.150971 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633cc316-e4b6-4e69-8a5f-d31ae13d274f None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:29.154776 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 95/196] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Aug 4 21:02:28 2022] POST /identity/v3/users => generated 297 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:02:29.155448 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908a492c-3e61-4cd1-81f8-494ff413c8ee None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:29.159812 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 102/197] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Aug 4 21:02:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:29.159812 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633cc316-e4b6-4e69-8a5f-d31ae13d274f None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:29.287784 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633cc316-e4b6-4e69-8a5f-d31ae13d274f None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:29.293047 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908a492c-3e61-4cd1-81f8-494ff413c8ee None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:29.326940 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38698388-5299-4056-bf8b-df2710e6d376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:29.327635 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38698388-5299-4056-bf8b-df2710e6d376 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:29.327916 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38698388-5299-4056-bf8b-df2710e6d376 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:29.328190 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38698388-5299-4056-bf8b-df2710e6d376 None admin] PATH_INFO: `/v3/users/40bb688e6e224290b1de6e4873bb7143` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:29.329358 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38698388-5299-4056-bf8b-df2710e6d376 None admin] RBAC: Authorizing `identity:get_user(user_id=40bb688e6e224290b1de6e4873bb7143)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:29.330807 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38698388-5299-4056-bf8b-df2710e6d376 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:29.332766 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 103/198] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Aug 4 21:02:29 2022] GET /identity/v3/users/40bb688e6e224290b1de6e4873bb7143 => generated 289 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:29.333293 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38698388-5299-4056-bf8b-df2710e6d376 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:29.333526 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f921777f-e585-4eff-90ef-b1e0c0aaeff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:29.334161 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f921777f-e585-4eff-90ef-b1e0c0aaeff5 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:29.334444 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f921777f-e585-4eff-90ef-b1e0c0aaeff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:29.334720 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f921777f-e585-4eff-90ef-b1e0c0aaeff5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:29.404882 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:29.414767 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:29.417539 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f921777f-e585-4eff-90ef-b1e0c0aaeff5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:29.419703 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f921777f-e585-4eff-90ef-b1e0c0aaeff5 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:29.458966 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38698388-5299-4056-bf8b-df2710e6d376 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:29.505324 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29feb19-f3ea-40b9-90b1-5188a3f7e107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:29.505934 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29feb19-f3ea-40b9-90b1-5188a3f7e107 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:29.506224 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29feb19-f3ea-40b9-90b1-5188a3f7e107 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:29.506505 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29feb19-f3ea-40b9-90b1-5188a3f7e107 None admin] PATH_INFO: `/v3/users/93d8deefa5254382b257e6e3fbe3d481` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:29.507829 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29feb19-f3ea-40b9-90b1-5188a3f7e107 None admin] RBAC: Authorizing `identity:get_user(user_id=93d8deefa5254382b257e6e3fbe3d481)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:29.509297 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29feb19-f3ea-40b9-90b1-5188a3f7e107 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:29.511226 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 104/199] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Aug 4 21:02:29 2022] GET /identity/v3/users/93d8deefa5254382b257e6e3fbe3d481 => generated 283 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:29.511959 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29feb19-f3ea-40b9-90b1-5188a3f7e107 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:29.549448 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 96/200] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Aug 4 21:02:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 21:02:29.551270 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f921777f-e585-4eff-90ef-b1e0c0aaeff5 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:29.621682 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29feb19-f3ea-40b9-90b1-5188a3f7e107 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:29.650703 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:29.654774 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:29.658813 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f921777f-e585-4eff-90ef-b1e0c0aaeff5 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:29.659730 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aa5cfb-213d-44a1-9c6f-bde56b5901a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:29.665683 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aa5cfb-213d-44a1-9c6f-bde56b5901a4 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:29.665683 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aa5cfb-213d-44a1-9c6f-bde56b5901a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:29.665683 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aa5cfb-213d-44a1-9c6f-bde56b5901a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:29.695922 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c55db72-f920-4f52-be83-596c673e7e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:29.697068 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c55db72-f920-4f52-be83-596c673e7e96 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:29.697514 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c55db72-f920-4f52-be83-596c673e7e96 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:29.698215 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c55db72-f920-4f52-be83-596c673e7e96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:29.735424 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64aa5cfb-213d-44a1-9c6f-bde56b5901a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:29.736927 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64aa5cfb-213d-44a1-9c6f-bde56b5901a4 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:29.786794 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c55db72-f920-4f52-be83-596c673e7e96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:29.788653 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c55db72-f920-4f52-be83-596c673e7e96 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:29.845156 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 105/201] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Aug 4 21:02:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 21:02:29.846112 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aa5cfb-213d-44a1-9c6f-bde56b5901a4 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:29.877543 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:29.929289 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 97/202] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Aug 4 21:02:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 21:02:29.930836 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c55db72-f920-4f52-be83-596c673e7e96 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:29.935721 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:29.964602 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aa5cfb-213d-44a1-9c6f-bde56b5901a4 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:30.011192 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f21cb7-23b6-4033-b0e0-b4402ca6b76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:30.012304 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f21cb7-23b6-4033-b0e0-b4402ca6b76a None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:30.012592 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f21cb7-23b6-4033-b0e0-b4402ca6b76a None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:30.013152 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f21cb7-23b6-4033-b0e0-b4402ca6b76a None admin] PATH_INFO: `/v3/projects/6c2de9abe1504f6698a9ea2a032c91e4` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:30.014611 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f21cb7-23b6-4033-b0e0-b4402ca6b76a None admin] RBAC: Authorizing `identity:get_project(project_id=6c2de9abe1504f6698a9ea2a032c91e4)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:30.016735 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f21cb7-23b6-4033-b0e0-b4402ca6b76a None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:30.019227 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 106/203] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Aug 4 21:02:29 2022] GET /identity/v3/projects/6c2de9abe1504f6698a9ea2a032c91e4 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:30.020156 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f21cb7-23b6-4033-b0e0-b4402ca6b76a None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:30.039137 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c55db72-f920-4f52-be83-596c673e7e96 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:30.077112 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caab9f5c-eeb3-4c31-9bec-7ff4b522708d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:30.078233 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caab9f5c-eeb3-4c31-9bec-7ff4b522708d None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:30.078531 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caab9f5c-eeb3-4c31-9bec-7ff4b522708d None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:30.078811 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caab9f5c-eeb3-4c31-9bec-7ff4b522708d None admin] PATH_INFO: `/v3/projects/b8f804a236884f039371233cce01ecef` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:30.080438 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caab9f5c-eeb3-4c31-9bec-7ff4b522708d None admin] RBAC: Authorizing `identity:get_project(project_id=b8f804a236884f039371233cce01ecef)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:30.082512 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caab9f5c-eeb3-4c31-9bec-7ff4b522708d None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:30.085007 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 98/204] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Aug 4 21:02:29 2022] GET /identity/v3/projects/b8f804a236884f039371233cce01ecef => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:30.090761 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caab9f5c-eeb3-4c31-9bec-7ff4b522708d None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:30.143032 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:30.148894 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f21cb7-23b6-4033-b0e0-b4402ca6b76a None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:30.183430 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:30.184682 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4498135b-4807-4c91-8eae-612d4eff5b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:30.185416 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4498135b-4807-4c91-8eae-612d4eff5b6c None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:30.185814 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4498135b-4807-4c91-8eae-612d4eff5b6c None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:30.186209 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4498135b-4807-4c91-8eae-612d4eff5b6c None admin] PATH_INFO: `/v3/projects/6c2de9abe1504f6698a9ea2a032c91e4` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:30.187485 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4498135b-4807-4c91-8eae-612d4eff5b6c None admin] RBAC: Authorizing `identity:get_project(project_id=6c2de9abe1504f6698a9ea2a032c91e4)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:30.190289 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4498135b-4807-4c91-8eae-612d4eff5b6c None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:30.192466 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 107/205] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Aug 4 21:02:30 2022] GET /identity/v3/projects/6c2de9abe1504f6698a9ea2a032c91e4 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:30.193367 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4498135b-4807-4c91-8eae-612d4eff5b6c None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:30.227568 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caab9f5c-eeb3-4c31-9bec-7ff4b522708d None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:30.268891 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b2256d-fb6b-4169-af2e-3235c3a43e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:30.269526 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b2256d-fb6b-4169-af2e-3235c3a43e01 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:30.269811 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b2256d-fb6b-4169-af2e-3235c3a43e01 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:30.270087 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b2256d-fb6b-4169-af2e-3235c3a43e01 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:30.271142 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b2256d-fb6b-4169-af2e-3235c3a43e01 None admin] RBAC: Adding query filter params (role.id=a432dd4735944ecfab2fa5ce1705d356, scope.project.id=6c2de9abe1504f6698a9ea2a032c91e4, user.id=93d8deefa5254382b257e6e3fbe3d481) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:30.271504 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b2256d-fb6b-4169-af2e-3235c3a43e01 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:30.273879 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b2256d-fb6b-4169-af2e-3235c3a43e01 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:30.280693 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 99/206] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Aug 4 21:02:30 2022] GET /identity/v3/role_assignments?user.id=93d8deefa5254382b257e6e3fbe3d481&scope.project.id=6c2de9abe1504f6698a9ea2a032c91e4&role.id=a432dd4735944ecfab2fa5ce1705d356 => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:30.281407 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b2256d-fb6b-4169-af2e-3235c3a43e01 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:30.308809 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4498135b-4807-4c91-8eae-612d4eff5b6c None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:30.348972 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7a5bdd-177e-42f8-a305-a442fedaf850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:30.350072 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7a5bdd-177e-42f8-a305-a442fedaf850 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:30.350706 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7a5bdd-177e-42f8-a305-a442fedaf850 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:30.351176 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7a5bdd-177e-42f8-a305-a442fedaf850 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:30.352328 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7a5bdd-177e-42f8-a305-a442fedaf850 None admin] RBAC: Adding query filter params (role.id=5ab0887d24bc41b882f6786b7aab0fe8, scope.project.id=b8f804a236884f039371233cce01ecef, user.id=40bb688e6e224290b1de6e4873bb7143) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:30.352856 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7a5bdd-177e-42f8-a305-a442fedaf850 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:30.354652 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7a5bdd-177e-42f8-a305-a442fedaf850 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:30.370571 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 108/207] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Aug 4 21:02:30 2022] GET /identity/v3/role_assignments?user.id=40bb688e6e224290b1de6e4873bb7143&scope.project.id=b8f804a236884f039371233cce01ecef&role.id=5ab0887d24bc41b882f6786b7aab0fe8 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:30.429432 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b2256d-fb6b-4169-af2e-3235c3a43e01 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:30.473626 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:30.479930 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6648f6a-815c-4ec7-aa66-7ff5a7b6b032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:30.480884 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6648f6a-815c-4ec7-aa66-7ff5a7b6b032 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:30.481328 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6648f6a-815c-4ec7-aa66-7ff5a7b6b032 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:30.481772 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6648f6a-815c-4ec7-aa66-7ff5a7b6b032 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:30.482796 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6648f6a-815c-4ec7-aa66-7ff5a7b6b032 None admin] RBAC: Adding query filter params (role.id=70587326d8584316b73b380df2ae8e67, scope.project.id=6c2de9abe1504f6698a9ea2a032c91e4, user.id=93d8deefa5254382b257e6e3fbe3d481) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:30.500321 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6648f6a-815c-4ec7-aa66-7ff5a7b6b032 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:30.500321 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6648f6a-815c-4ec7-aa66-7ff5a7b6b032 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:30.500321 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 100/208] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Aug 4 21:02:30 2022] GET /identity/v3/role_assignments?user.id=93d8deefa5254382b257e6e3fbe3d481&scope.project.id=6c2de9abe1504f6698a9ea2a032c91e4&role.id=70587326d8584316b73b380df2ae8e67 => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:31.467479 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3903cedd-6f87-4eb7-8ec2-065990b3f70b None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:31.468015 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3903cedd-6f87-4eb7-8ec2-065990b3f70b None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:31.468093 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3903cedd-6f87-4eb7-8ec2-065990b3f70b None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:31.469622 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 109/209] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:02:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:02:31.475084 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72cf8f0-ad6f-4527-8eef-5ebf0ca1ea3c None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:31.475969 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72cf8f0-ad6f-4527-8eef-5ebf0ca1ea3c None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:31.476460 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72cf8f0-ad6f-4527-8eef-5ebf0ca1ea3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:31.495544 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-f72cf8f0-ad6f-4527-8eef-5ebf0ca1ea3c None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:31.497038 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-f72cf8f0-ad6f-4527-8eef-5ebf0ca1ea3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:31.515289 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c5d30-d730-43c4-89a1-be82116cc555 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:31.516122 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c5d30-d730-43c4-89a1-be82116cc555 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:31.516592 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c5d30-d730-43c4-89a1-be82116cc555 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:31.518757 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 110/210] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:02:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:02:31.525808 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de8321f-3094-46ce-bed2-9f94e57cf8dd None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:31.526228 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de8321f-3094-46ce-bed2-9f94e57cf8dd None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:31.526634 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de8321f-3094-46ce-bed2-9f94e57cf8dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:31.550993 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-8de8321f-3094-46ce-bed2-9f94e57cf8dd None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:31.552633 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-8de8321f-3094-46ce-bed2-9f94e57cf8dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:31.583444 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 101/211] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:02:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 21:02:31.587667 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:31.603522 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72cf8f0-ad6f-4527-8eef-5ebf0ca1ea3c None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:31.635368 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 111/212] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:02:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:02:31.656924 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de8321f-3094-46ce-bed2-9f94e57cf8dd None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:31.728530 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72cf8f0-ad6f-4527-8eef-5ebf0ca1ea3c None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:31.764630 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:31.773305 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de8321f-3094-46ce-bed2-9f94e57cf8dd None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:31.776767 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579758ac-3ddd-4023-aa36-0ff6ee9e7938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:31.778298 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579758ac-3ddd-4023-aa36-0ff6ee9e7938 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:31.778298 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579758ac-3ddd-4023-aa36-0ff6ee9e7938 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:31.778298 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579758ac-3ddd-4023-aa36-0ff6ee9e7938 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:31.794673 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579758ac-3ddd-4023-aa36-0ff6ee9e7938 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:02:31.799676 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-579758ac-3ddd-4023-aa36-0ff6ee9e7938 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 21:02:31.799676 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:02:31.799676 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:02:31.799676 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:02:31.799676 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:02:31.799676 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:02:31.799676 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:02:31.799676 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:02:31.799676 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:02:31.799676 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:02:31.799676 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:02:31.799676 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:02:31.799676 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:02:31.799676 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:02:31.799676 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:02:31.799676 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:02:31.801141 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:02:31.801141 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:02:31.801141 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:02:31.801141 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:02:31.801141 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:02:31.801141 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:02:31.801141 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:02:31.801141 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:02:31.801141 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:02:31.801141 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:02:31.801141 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:02:31.801141 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:02:31.801141 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:02:31.801141 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:02:31.801141 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:02:31.801141 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:02:31.801922 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:02:31.801922 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:02:31.801922 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:02:31.801922 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:02:31.801922 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:02:31.801922 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:02:31.801922 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:02:31.801922 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:02:31.801922 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:02:31.801922 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:02:31.801922 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 21:02:31.801922 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:02:31.801922 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 102/213] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Aug 4 21:02:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:02:31.808953 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579758ac-3ddd-4023-aa36-0ff6ee9e7938 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:31.826549 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373ef0f3-b209-4e8f-8314-a6c6f9635049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:31.827528 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373ef0f3-b209-4e8f-8314-a6c6f9635049 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:31.827984 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373ef0f3-b209-4e8f-8314-a6c6f9635049 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:31.828611 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373ef0f3-b209-4e8f-8314-a6c6f9635049 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:31.830290 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373ef0f3-b209-4e8f-8314-a6c6f9635049 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:31.831968 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373ef0f3-b209-4e8f-8314-a6c6f9635049 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:31.834523 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 112/214] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:02:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:02:31.841856 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373ef0f3-b209-4e8f-8314-a6c6f9635049 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:31.874945 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:31.947470 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579758ac-3ddd-4023-aa36-0ff6ee9e7938 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:31.951196 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373ef0f3-b209-4e8f-8314-a6c6f9635049 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:31.988351 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336afede-2ac8-470b-94ff-bd61bac465ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:31.989049 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336afede-2ac8-470b-94ff-bd61bac465ed None admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:31.989432 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336afede-2ac8-470b-94ff-bd61bac465ed None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:31.989807 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336afede-2ac8-470b-94ff-bd61bac465ed None admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:31.990611 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336afede-2ac8-470b-94ff-bd61bac465ed None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:31.992328 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336afede-2ac8-470b-94ff-bd61bac465ed None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:31.993933 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b9689d-ea9f-47a3-b1c0-3e3a1ec6fd03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:31.996486 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b9689d-ea9f-47a3-b1c0-3e3a1ec6fd03 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:31.996486 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b9689d-ea9f-47a3-b1c0-3e3a1ec6fd03 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:31.996486 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b9689d-ea9f-47a3-b1c0-3e3a1ec6fd03 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:31.996486 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b9689d-ea9f-47a3-b1c0-3e3a1ec6fd03 None admin] RBAC: Adding query filter params (name=member) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:31.996486 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b9689d-ea9f-47a3-b1c0-3e3a1ec6fd03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:31.998710 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b9689d-ea9f-47a3-b1c0-3e3a1ec6fd03 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:32.008985 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 103/215] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Aug 4 21:02:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:02:32.014802 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b9689d-ea9f-47a3-b1c0-3e3a1ec6fd03 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:32.067414 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:32.107098 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 113/216] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Aug 4 21:02:31 2022] POST /identity/v3/users => generated 297 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:02:32.131258 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b9689d-ea9f-47a3-b1c0-3e3a1ec6fd03 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:32.175562 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f295c010-bfaf-4633-95db-a553bcea3075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:32.176659 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f295c010-bfaf-4633-95db-a553bcea3075 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:32.177191 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f295c010-bfaf-4633-95db-a553bcea3075 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:32.177663 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f295c010-bfaf-4633-95db-a553bcea3075 None admin] PATH_INFO: `/v3/users/c43671b71d544cd983ba8d656fa457e3` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:32.194712 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f295c010-bfaf-4633-95db-a553bcea3075 None admin] RBAC: Authorizing `identity:get_user(user_id=c43671b71d544cd983ba8d656fa457e3)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:32.196870 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f295c010-bfaf-4633-95db-a553bcea3075 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:32.198935 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 104/217] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Aug 4 21:02:32 2022] GET /identity/v3/users/c43671b71d544cd983ba8d656fa457e3 => generated 297 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:02:32.205379 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336afede-2ac8-470b-94ff-bd61bac465ed None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:32.308901 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:32.327826 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336afede-2ac8-470b-94ff-bd61bac465ed None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:32.365451 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3f60ba-49af-438c-bade-d3f242ecd704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:32.366473 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3f60ba-49af-438c-bade-d3f242ecd704 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:32.366956 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3f60ba-49af-438c-bade-d3f242ecd704 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:32.367532 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3f60ba-49af-438c-bade-d3f242ecd704 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:32.442389 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3f60ba-49af-438c-bade-d3f242ecd704 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:32.444188 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3f60ba-49af-438c-bade-d3f242ecd704 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:32.540127 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:32.560647 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 114/218] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Aug 4 21:02:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 21:02:32.566357 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f295c010-bfaf-4633-95db-a553bcea3075 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:32.583961 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c186a428-8d5b-4270-a4e8-1595927247a8 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:32.584258 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c186a428-8d5b-4270-a4e8-1595927247a8 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:32.584543 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c186a428-8d5b-4270-a4e8-1595927247a8 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:32.586421 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 115/219] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:02:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 21:02:32.592776 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953c4f78-71e0-4490-8cf8-37bac881cbff None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:32.593064 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953c4f78-71e0-4490-8cf8-37bac881cbff None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:32.593345 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953c4f78-71e0-4490-8cf8-37bac881cbff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:32.615015 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-953c4f78-71e0-4490-8cf8-37bac881cbff None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:32.616493 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-953c4f78-71e0-4490-8cf8-37bac881cbff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:32.691914 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 116/220] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:02:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:02:32.692999 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0aff63-8c74-4fe6-9c8a-8c57c6340c9c None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:32.699426 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0aff63-8c74-4fe6-9c8a-8c57c6340c9c None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:32.699426 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0aff63-8c74-4fe6-9c8a-8c57c6340c9c None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:32.699426 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 117/221] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:02:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:02:32.700456 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0aa296-4476-457d-8c14-526bb3b97d9a None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:32.700629 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0aa296-4476-457d-8c14-526bb3b97d9a None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:32.701035 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0aa296-4476-457d-8c14-526bb3b97d9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:32.703098 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f295c010-bfaf-4633-95db-a553bcea3075 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:32.713214 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:32.726821 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-8d0aa296-4476-457d-8c14-526bb3b97d9a None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:32.729967 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-8d0aa296-4476-457d-8c14-526bb3b97d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:32.755705 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8782bd7-dba1-474b-93b0-a2b4646e8db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:32.756352 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8782bd7-dba1-474b-93b0-a2b4646e8db2 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:32.756633 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8782bd7-dba1-474b-93b0-a2b4646e8db2 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:32.756907 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8782bd7-dba1-474b-93b0-a2b4646e8db2 None admin] PATH_INFO: `/v3/projects/6c2de9abe1504f6698a9ea2a032c91e4` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:32.758109 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8782bd7-dba1-474b-93b0-a2b4646e8db2 None admin] RBAC: Authorizing `identity:get_project(project_id=6c2de9abe1504f6698a9ea2a032c91e4)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:32.759717 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8782bd7-dba1-474b-93b0-a2b4646e8db2 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:32.761375 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 105/222] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Aug 4 21:02:32 2022] GET /identity/v3/projects/6c2de9abe1504f6698a9ea2a032c91e4 => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:32.762249 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8782bd7-dba1-474b-93b0-a2b4646e8db2 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:32.819680 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 118/223] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:02:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 21:02:32.828113 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0aa296-4476-457d-8c14-526bb3b97d9a None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:32.884614 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:32.889292 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8782bd7-dba1-474b-93b0-a2b4646e8db2 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:32.926740 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c8ca44-8b5d-418a-90e9-e3100f457c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:32.927428 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c8ca44-8b5d-418a-90e9-e3100f457c1d None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:32.927707 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c8ca44-8b5d-418a-90e9-e3100f457c1d None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:32.927982 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c8ca44-8b5d-418a-90e9-e3100f457c1d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:32.939079 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c8ca44-8b5d-418a-90e9-e3100f457c1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:02:32.944325 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-79c8ca44-8b5d-418a-90e9-e3100f457c1d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:02:32.944325 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:02:32.944325 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:02:32.944325 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:02:32.944325 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:02:32.944325 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:02:32.944325 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:02:32.944325 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:02:32.944325 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:02:32.944325 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:02:32.944325 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:02:32.944325 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:02:32.944325 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:02:32.944325 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:02:32.944325 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:02:32.944325 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:02:32.945235 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:02:32.945235 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:02:32.945235 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:02:32.945235 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:02:32.945235 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:02:32.945235 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:02:32.945235 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:02:32.945235 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:02:32.945235 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:02:32.945235 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:02:32.945235 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:02:32.945235 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:02:32.945235 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:02:32.945235 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:02:32.945235 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:02:32.945235 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:02:32.946043 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:02:32.946043 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:02:32.946043 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:02:32.946043 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:02:32.946043 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:02:32.946043 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:02:32.946043 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:02:32.946043 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:02:32.946043 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:02:32.946043 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:02:32.946043 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:02:32.946043 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:02:32.946043 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 106/224] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Aug 4 21:02:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:02:32.946713 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c8ca44-8b5d-418a-90e9-e3100f457c1d None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:32.965121 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0aa296-4476-457d-8c14-526bb3b97d9a None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:32.983247 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:33.006101 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1767fe88-45ea-413b-93dc-a2bbeddfdd1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:33.007041 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1767fe88-45ea-413b-93dc-a2bbeddfdd1f None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:33.007504 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1767fe88-45ea-413b-93dc-a2bbeddfdd1f None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:33.007957 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1767fe88-45ea-413b-93dc-a2bbeddfdd1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:33.008908 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1767fe88-45ea-413b-93dc-a2bbeddfdd1f None admin] RBAC: Adding query filter params (role.id=8feebc60363c4feb870718b1721e5324, scope.project.id=6c2de9abe1504f6698a9ea2a032c91e4, user.id=c43671b71d544cd983ba8d656fa457e3) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:33.009367 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1767fe88-45ea-413b-93dc-a2bbeddfdd1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:33.010976 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1767fe88-45ea-413b-93dc-a2bbeddfdd1f None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:33.017208 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 119/225] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Aug 4 21:02:32 2022] GET /identity/v3/role_assignments?user.id=c43671b71d544cd983ba8d656fa457e3&scope.project.id=6c2de9abe1504f6698a9ea2a032c91e4&role.id=8feebc60363c4feb870718b1721e5324 => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:33.018225 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1767fe88-45ea-413b-93dc-a2bbeddfdd1f None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:33.057539 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c8ca44-8b5d-418a-90e9-e3100f457c1d None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:33.095959 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d845cbb-07cf-4895-97dd-89248be9955c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:33.096903 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d845cbb-07cf-4895-97dd-89248be9955c None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:33.097021 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d845cbb-07cf-4895-97dd-89248be9955c None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:33.097230 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d845cbb-07cf-4895-97dd-89248be9955c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:33.109035 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d845cbb-07cf-4895-97dd-89248be9955c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:02:33.114546 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-9d845cbb-07cf-4895-97dd-89248be9955c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 21:02:33.114546 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:02:33.114546 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:02:33.114546 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:02:33.114546 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:02:33.114546 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:02:33.114546 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:02:33.114546 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:02:33.114546 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:02:33.114546 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:02:33.114546 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:02:33.114546 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:02:33.114546 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:02:33.114546 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:02:33.114546 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:02:33.114546 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:02:33.115568 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:02:33.115568 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:02:33.115568 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:02:33.115568 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:02:33.115568 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:02:33.115568 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:02:33.115568 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:02:33.115568 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:02:33.115568 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:02:33.115568 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:02:33.115568 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:02:33.115568 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:02:33.115568 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:02:33.115568 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:02:33.115568 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:02:33.115568 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:02:33.116357 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:02:33.116357 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:02:33.116357 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:02:33.116357 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:02:33.116357 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:02:33.116357 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:02:33.116357 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:02:33.116357 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:02:33.116357 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:02:33.116357 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:02:33.116357 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 21:02:33.116357 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:02:33.116357 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 107/226] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Aug 4 21:02:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:02:33.123629 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d845cbb-07cf-4895-97dd-89248be9955c None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:33.158038 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1767fe88-45ea-413b-93dc-a2bbeddfdd1f None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:33.171291 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:33.238404 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a53004-ee7e-436f-994e-7ee7e349f2d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:33.238904 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a53004-ee7e-436f-994e-7ee7e349f2d8 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:33.239194 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a53004-ee7e-436f-994e-7ee7e349f2d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:33.239595 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a53004-ee7e-436f-994e-7ee7e349f2d8 None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:33.240340 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a53004-ee7e-436f-994e-7ee7e349f2d8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:33.240659 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a53004-ee7e-436f-994e-7ee7e349f2d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:33.242051 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a53004-ee7e-436f-994e-7ee7e349f2d8 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:33.251681 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:33.256530 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d845cbb-07cf-4895-97dd-89248be9955c None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:33.257187 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 120/227] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Aug 4 21:02:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:33.258510 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a53004-ee7e-436f-994e-7ee7e349f2d8 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:33.300065 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daea64c-fc11-46cb-b35b-0d7ff6a3e2fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:33.301080 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daea64c-fc11-46cb-b35b-0d7ff6a3e2fa None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:33.301897 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daea64c-fc11-46cb-b35b-0d7ff6a3e2fa None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:33.302386 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daea64c-fc11-46cb-b35b-0d7ff6a3e2fa None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:33.303478 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daea64c-fc11-46cb-b35b-0d7ff6a3e2fa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:33.303983 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daea64c-fc11-46cb-b35b-0d7ff6a3e2fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:33.305730 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daea64c-fc11-46cb-b35b-0d7ff6a3e2fa None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:33.315486 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 108/228] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Aug 4 21:02:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:33.324736 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daea64c-fc11-46cb-b35b-0d7ff6a3e2fa None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:33.444958 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a53004-ee7e-436f-994e-7ee7e349f2d8 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:33.465297 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daea64c-fc11-46cb-b35b-0d7ff6a3e2fa None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:33.506986 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c268f6-981e-4e5e-a649-3511efe797bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:33.507672 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c268f6-981e-4e5e-a649-3511efe797bc None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:33.507958 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c268f6-981e-4e5e-a649-3511efe797bc None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:33.508238 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c268f6-981e-4e5e-a649-3511efe797bc None admin] PATH_INFO: `/v3/users/93d8deefa5254382b257e6e3fbe3d481` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:33.509480 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c268f6-981e-4e5e-a649-3511efe797bc None admin] RBAC: Authorizing `identity:get_user(user_id=93d8deefa5254382b257e6e3fbe3d481)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:33.510940 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c268f6-981e-4e5e-a649-3511efe797bc None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:33.517302 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 121/229] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Aug 4 21:02:33 2022] GET /identity/v3/users/93d8deefa5254382b257e6e3fbe3d481 => generated 283 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:33.518576 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c268f6-981e-4e5e-a649-3511efe797bc None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:33.549468 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264cbea6-6d2d-4bb7-a174-8a3c4fd0b1a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:33.550661 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264cbea6-6d2d-4bb7-a174-8a3c4fd0b1a8 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:33.550949 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264cbea6-6d2d-4bb7-a174-8a3c4fd0b1a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:33.551227 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264cbea6-6d2d-4bb7-a174-8a3c4fd0b1a8 None admin] PATH_INFO: `/v3/users/93d8deefa5254382b257e6e3fbe3d481` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:33.556037 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264cbea6-6d2d-4bb7-a174-8a3c4fd0b1a8 None admin] RBAC: Authorizing `identity:get_user(user_id=93d8deefa5254382b257e6e3fbe3d481)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:33.556544 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:33.557541 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264cbea6-6d2d-4bb7-a174-8a3c4fd0b1a8 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:33.559183 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 109/230] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Aug 4 21:02:33 2022] GET /identity/v3/users/93d8deefa5254382b257e6e3fbe3d481 => generated 283 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:33.564787 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264cbea6-6d2d-4bb7-a174-8a3c4fd0b1a8 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:33.627800 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:33.679128 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264cbea6-6d2d-4bb7-a174-8a3c4fd0b1a8 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:33.681366 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c268f6-981e-4e5e-a649-3511efe797bc None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:33.716774 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1ea70f-615f-425c-a6c2-a5793dbf3a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:33.717780 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1ea70f-615f-425c-a6c2-a5793dbf3a8d None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:33.718260 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1ea70f-615f-425c-a6c2-a5793dbf3a8d None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:33.718753 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1ea70f-615f-425c-a6c2-a5793dbf3a8d None admin] PATH_INFO: `/v3/projects/6c2de9abe1504f6698a9ea2a032c91e4` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:33.720160 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1ea70f-615f-425c-a6c2-a5793dbf3a8d None admin] RBAC: Authorizing `identity:get_project(project_id=6c2de9abe1504f6698a9ea2a032c91e4)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:33.721915 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1ea70f-615f-425c-a6c2-a5793dbf3a8d None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:33.723859 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 110/231] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Aug 4 21:02:33 2022] GET /identity/v3/projects/6c2de9abe1504f6698a9ea2a032c91e4 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:02:33.729418 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1ea70f-615f-425c-a6c2-a5793dbf3a8d None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:33.817363 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2faaff5-01b9-4c1d-a94d-d797504706ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:33.818031 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2faaff5-01b9-4c1d-a94d-d797504706ea None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:33.818315 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2faaff5-01b9-4c1d-a94d-d797504706ea None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:33.818593 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2faaff5-01b9-4c1d-a94d-d797504706ea None admin] PATH_INFO: `/v3/projects/6c2de9abe1504f6698a9ea2a032c91e4` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:33.824084 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2faaff5-01b9-4c1d-a94d-d797504706ea None admin] RBAC: Authorizing `identity:get_project(project_id=6c2de9abe1504f6698a9ea2a032c91e4)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:33.825537 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2faaff5-01b9-4c1d-a94d-d797504706ea None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:33.827604 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 122/232] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Aug 4 21:02:33 2022] GET /identity/v3/projects/6c2de9abe1504f6698a9ea2a032c91e4 => generated 302 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:02:33.836202 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2faaff5-01b9-4c1d-a94d-d797504706ea None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:33.837297 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1ea70f-615f-425c-a6c2-a5793dbf3a8d None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:33.849689 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:33.874140 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5576f3-ac17-4864-a841-5c8ba4cf8795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:33.875256 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5576f3-ac17-4864-a841-5c8ba4cf8795 None admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:33.875768 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5576f3-ac17-4864-a841-5c8ba4cf8795 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:33.876353 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5576f3-ac17-4864-a841-5c8ba4cf8795 None admin] PATH_INFO: `/v3/projects/6c2de9abe1504f6698a9ea2a032c91e4/users/93d8deefa5254382b257e6e3fbe3d481/roles/70587326d8584316b73b380df2ae8e67` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:33.879533 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5576f3-ac17-4864-a841-5c8ba4cf8795 None admin] RBAC: Authorizing `identity:create_grant(project_id=6c2de9abe1504f6698a9ea2a032c91e4, user_id=93d8deefa5254382b257e6e3fbe3d481, role_id=70587326d8584316b73b380df2ae8e67)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:33.881938 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5576f3-ac17-4864-a841-5c8ba4cf8795 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:33.958811 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 111/233] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Aug 4 21:02:33 2022] PUT /identity/v3/projects/6c2de9abe1504f6698a9ea2a032c91e4/users/93d8deefa5254382b257e6e3fbe3d481/roles/70587326d8584316b73b380df2ae8e67 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:02:33.974530 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2faaff5-01b9-4c1d-a94d-d797504706ea None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:33.977202 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:34.011499 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5039e996-f556-4cc9-8415-8eacc72a22d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:34.012184 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5039e996-f556-4cc9-8415-8eacc72a22d3 None admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:34.012472 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5039e996-f556-4cc9-8415-8eacc72a22d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:34.012751 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5039e996-f556-4cc9-8415-8eacc72a22d3 None admin] PATH_INFO: `/v3/projects/6c2de9abe1504f6698a9ea2a032c91e4/users/93d8deefa5254382b257e6e3fbe3d481/roles/a432dd4735944ecfab2fa5ce1705d356` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:34.014753 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5039e996-f556-4cc9-8415-8eacc72a22d3 None admin] RBAC: Authorizing `identity:create_grant(project_id=6c2de9abe1504f6698a9ea2a032c91e4, user_id=93d8deefa5254382b257e6e3fbe3d481, role_id=a432dd4735944ecfab2fa5ce1705d356)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:34.016450 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5039e996-f556-4cc9-8415-8eacc72a22d3 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:34.103797 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 123/234] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Aug 4 21:02:33 2022] PUT /identity/v3/projects/6c2de9abe1504f6698a9ea2a032c91e4/users/93d8deefa5254382b257e6e3fbe3d481/roles/a432dd4735944ecfab2fa5ce1705d356 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:02:34.665587 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5083510-565a-47a8-abf6-5e4c810d8bf5 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:34.666609 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5083510-565a-47a8-abf6-5e4c810d8bf5 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:34.667141 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5083510-565a-47a8-abf6-5e4c810d8bf5 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:34.668752 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 112/235] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:02:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:02:34.674235 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26884714-9547-4b92-8358-5f211d36c95e None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:34.675116 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26884714-9547-4b92-8358-5f211d36c95e None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:34.675633 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26884714-9547-4b92-8358-5f211d36c95e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:34.700263 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-26884714-9547-4b92-8358-5f211d36c95e None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:34.701783 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-26884714-9547-4b92-8358-5f211d36c95e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:34.715925 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf7bda3-112c-480b-8ab9-91819f970012 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:34.716206 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf7bda3-112c-480b-8ab9-91819f970012 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:34.716491 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf7bda3-112c-480b-8ab9-91819f970012 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:34.718296 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 113/236] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:02:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:02:34.724837 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e035f343-0d8e-4554-b88b-4b2d42231402 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:34.725129 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e035f343-0d8e-4554-b88b-4b2d42231402 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:34.725411 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e035f343-0d8e-4554-b88b-4b2d42231402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:34.748204 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-e035f343-0d8e-4554-b88b-4b2d42231402 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:34.749703 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-e035f343-0d8e-4554-b88b-4b2d42231402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:34.824450 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 124/237] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:02:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:02:34.845305 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 114/238] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:02:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 21:02:34.860014 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26884714-9547-4b92-8358-5f211d36c95e None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:34.892783 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e035f343-0d8e-4554-b88b-4b2d42231402 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:34.998826 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26884714-9547-4b92-8358-5f211d36c95e None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:35.001761 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:35.035310 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d1636e-f3f6-40bb-bbcd-8209cf878196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:35.035991 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d1636e-f3f6-40bb-bbcd-8209cf878196 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:35.036274 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d1636e-f3f6-40bb-bbcd-8209cf878196 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:35.036548 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d1636e-f3f6-40bb-bbcd-8209cf878196 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:35.037784 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d1636e-f3f6-40bb-bbcd-8209cf878196 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:35.039263 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d1636e-f3f6-40bb-bbcd-8209cf878196 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:35.040910 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 125/239] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:02:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:35.041608 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:35.046315 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d1636e-f3f6-40bb-bbcd-8209cf878196 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:35.082061 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e035f343-0d8e-4554-b88b-4b2d42231402 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:35.148000 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56d3fc2-a11e-43d9-9bc6-7c4a198c5a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:35.149048 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56d3fc2-a11e-43d9-9bc6-7c4a198c5a5d None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:35.149524 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56d3fc2-a11e-43d9-9bc6-7c4a198c5a5d None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:35.150011 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56d3fc2-a11e-43d9-9bc6-7c4a198c5a5d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:35.151235 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d1636e-f3f6-40bb-bbcd-8209cf878196 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:35.176444 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56d3fc2-a11e-43d9-9bc6-7c4a198c5a5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:02:35.182008 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-f56d3fc2-a11e-43d9-9bc6-7c4a198c5a5d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 21:02:35.182008 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:02:35.182008 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:02:35.182008 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:02:35.182008 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:02:35.182008 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:02:35.182008 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:02:35.182008 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:02:35.182008 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:02:35.182008 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:02:35.182008 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:02:35.182008 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:02:35.182008 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:02:35.182008 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:02:35.182008 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:02:35.182008 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:02:35.182865 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:02:35.182865 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:02:35.182865 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:02:35.182865 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:02:35.182865 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:02:35.182865 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:02:35.182865 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:02:35.182865 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:02:35.182865 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:02:35.182865 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:02:35.182865 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:02:35.182865 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:02:35.182865 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:02:35.182865 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:02:35.182865 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:02:35.182865 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:02:35.192140 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:02:35.192140 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:02:35.192140 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:02:35.192140 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:02:35.192140 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:02:35.192140 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:02:35.192140 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:02:35.192140 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:02:35.192140 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:02:35.192140 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:02:35.192140 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 21:02:35.192140 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:02:35.192140 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 115/240] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Aug 4 21:02:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:02:35.194929 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56d3fc2-a11e-43d9-9bc6-7c4a198c5a5d None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:35.198922 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009edcd5-7907-48c6-8899-8f68d7fd236e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:35.199614 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009edcd5-7907-48c6-8899-8f68d7fd236e None admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:35.199901 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009edcd5-7907-48c6-8899-8f68d7fd236e None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:35.200176 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009edcd5-7907-48c6-8899-8f68d7fd236e None admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:35.200893 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009edcd5-7907-48c6-8899-8f68d7fd236e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:35.202472 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009edcd5-7907-48c6-8899-8f68d7fd236e None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:35.293933 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 126/241] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Aug 4 21:02:35 2022] POST /identity/v3/users => generated 293 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:02:35.343634 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56d3fc2-a11e-43d9-9bc6-7c4a198c5a5d None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:35.392926 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b2d71a-74b5-42c2-b626-1802d64d1a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:35.393655 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b2d71a-74b5-42c2-b626-1802d64d1a93 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:35.394451 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b2d71a-74b5-42c2-b626-1802d64d1a93 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:35.394725 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b2d71a-74b5-42c2-b626-1802d64d1a93 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:35.395489 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b2d71a-74b5-42c2-b626-1802d64d1a93 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:35.395797 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b2d71a-74b5-42c2-b626-1802d64d1a93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:35.397212 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b2d71a-74b5-42c2-b626-1802d64d1a93 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:35.411213 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 116/242] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Aug 4 21:02:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:35.417090 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009edcd5-7907-48c6-8899-8f68d7fd236e None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:35.448045 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:35.569039 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009edcd5-7907-48c6-8899-8f68d7fd236e None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:35.606732 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043b6fc5-1368-446d-b7a5-ffc2a12927a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:35.607478 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043b6fc5-1368-446d-b7a5-ffc2a12927a6 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:35.607818 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043b6fc5-1368-446d-b7a5-ffc2a12927a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:35.608100 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043b6fc5-1368-446d-b7a5-ffc2a12927a6 None admin] PATH_INFO: `/v3/users/d360e0a89a50403aa353066d2b7fe68d` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:35.624686 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafdaffc-4809-43df-a96f-e55734cf3f51 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:35.624993 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafdaffc-4809-43df-a96f-e55734cf3f51 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:35.625259 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafdaffc-4809-43df-a96f-e55734cf3f51 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:35.628105 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 117/243] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:02:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:02:35.628818 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043b6fc5-1368-446d-b7a5-ffc2a12927a6 None admin] RBAC: Authorizing `identity:get_user(user_id=d360e0a89a50403aa353066d2b7fe68d)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:35.630176 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043b6fc5-1368-446d-b7a5-ffc2a12927a6 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:35.632104 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 127/244] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Aug 4 21:02:35 2022] GET /identity/v3/users/d360e0a89a50403aa353066d2b7fe68d => generated 297 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:35.634973 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f24d608-1936-46c9-8ade-e5aa28d91c3e None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:35.635269 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f24d608-1936-46c9-8ade-e5aa28d91c3e None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:35.635580 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f24d608-1936-46c9-8ade-e5aa28d91c3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:35.644026 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043b6fc5-1368-446d-b7a5-ffc2a12927a6 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:35.645998 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:35.658985 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-3f24d608-1936-46c9-8ade-e5aa28d91c3e None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:35.659478 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-3f24d608-1936-46c9-8ade-e5aa28d91c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:35.749560 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:35.762997 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 118/245] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:02:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:02:35.780811 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043b6fc5-1368-446d-b7a5-ffc2a12927a6 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:35.787004 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f24d608-1936-46c9-8ade-e5aa28d91c3e None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:35.829361 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c958e85-2ac1-48cc-ad4c-46b3e022dd3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:35.829919 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c958e85-2ac1-48cc-ad4c-46b3e022dd3a None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:35.830204 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c958e85-2ac1-48cc-ad4c-46b3e022dd3a None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:35.830480 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c958e85-2ac1-48cc-ad4c-46b3e022dd3a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:35.929934 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c958e85-2ac1-48cc-ad4c-46b3e022dd3a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:35.932835 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c958e85-2ac1-48cc-ad4c-46b3e022dd3a None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:35.935528 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f24d608-1936-46c9-8ade-e5aa28d91c3e None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:35.989372 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f545b8-bb57-462d-9468-6d45570c5642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:35.990469 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f545b8-bb57-462d-9468-6d45570c5642 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:35.990946 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f545b8-bb57-462d-9468-6d45570c5642 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:35.991440 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f545b8-bb57-462d-9468-6d45570c5642 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:35.994474 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:36.005746 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f545b8-bb57-462d-9468-6d45570c5642 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:02:36.012055 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-78f545b8-bb57-462d-9468-6d45570c5642 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 21:02:36.012055 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:02:36.012055 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:02:36.012055 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:02:36.012055 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:02:36.012055 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:02:36.012055 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:02:36.012055 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:02:36.012055 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:02:36.012055 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:02:36.012055 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:02:36.012055 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:02:36.012055 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:02:36.012055 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:02:36.012055 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:02:36.012055 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:02:36.012913 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:02:36.012913 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:02:36.012913 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:02:36.012913 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:02:36.012913 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:02:36.012913 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:02:36.012913 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:02:36.012913 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:02:36.012913 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:02:36.012913 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:02:36.012913 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:02:36.012913 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:02:36.012913 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:02:36.012913 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:02:36.012913 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:02:36.012913 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:02:36.013805 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:02:36.013805 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:02:36.013805 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:02:36.013805 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:02:36.013805 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:02:36.013805 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:02:36.013805 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:02:36.013805 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:02:36.013805 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:02:36.013805 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:02:36.013805 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 21:02:36.013805 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:02:36.016397 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 119/246] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Aug 4 21:02:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:02:36.023084 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f545b8-bb57-462d-9468-6d45570c5642 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:36.036734 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:36.081290 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 128/247] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Aug 4 21:02:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 21:02:36.088444 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c958e85-2ac1-48cc-ad4c-46b3e022dd3a None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:36.163755 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f545b8-bb57-462d-9468-6d45570c5642 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:36.216502 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18873701-7c0e-4ffb-85ee-fcf9e65d1085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:36.217135 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18873701-7c0e-4ffb-85ee-fcf9e65d1085 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:36.217420 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18873701-7c0e-4ffb-85ee-fcf9e65d1085 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:36.222380 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18873701-7c0e-4ffb-85ee-fcf9e65d1085 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:36.222380 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18873701-7c0e-4ffb-85ee-fcf9e65d1085 None admin] RBAC: Adding query filter params (name=member) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:36.222380 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18873701-7c0e-4ffb-85ee-fcf9e65d1085 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:36.222380 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18873701-7c0e-4ffb-85ee-fcf9e65d1085 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:36.234005 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 120/248] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Aug 4 21:02:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:36.242021 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18873701-7c0e-4ffb-85ee-fcf9e65d1085 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:36.260406 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c958e85-2ac1-48cc-ad4c-46b3e022dd3a None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:36.320262 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068109a6-4c20-4591-9a55-bc41171895f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:36.321289 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068109a6-4c20-4591-9a55-bc41171895f1 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:36.323026 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068109a6-4c20-4591-9a55-bc41171895f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:36.323521 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068109a6-4c20-4591-9a55-bc41171895f1 None admin] PATH_INFO: `/v3/projects/6c2de9abe1504f6698a9ea2a032c91e4` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:36.324896 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068109a6-4c20-4591-9a55-bc41171895f1 None admin] RBAC: Authorizing `identity:get_project(project_id=6c2de9abe1504f6698a9ea2a032c91e4)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:36.327206 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068109a6-4c20-4591-9a55-bc41171895f1 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:36.332021 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 129/249] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Aug 4 21:02:36 2022] GET /identity/v3/projects/6c2de9abe1504f6698a9ea2a032c91e4 => generated 302 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:02:36.340272 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068109a6-4c20-4591-9a55-bc41171895f1 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:36.360037 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:36.423491 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:36.439858 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18873701-7c0e-4ffb-85ee-fcf9e65d1085 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:36.509437 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068109a6-4c20-4591-9a55-bc41171895f1 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:36.510998 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7777b7e-6e61-4b78-8622-32e08703e5ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:36.511692 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7777b7e-6e61-4b78-8622-32e08703e5ec None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:36.511980 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7777b7e-6e61-4b78-8622-32e08703e5ec None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:36.512255 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7777b7e-6e61-4b78-8622-32e08703e5ec None admin] PATH_INFO: `/v3/users/c43671b71d544cd983ba8d656fa457e3` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:36.513469 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7777b7e-6e61-4b78-8622-32e08703e5ec None admin] RBAC: Authorizing `identity:get_user(user_id=c43671b71d544cd983ba8d656fa457e3)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:36.514958 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7777b7e-6e61-4b78-8622-32e08703e5ec None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:36.516645 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 121/250] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Aug 4 21:02:36 2022] GET /identity/v3/users/c43671b71d544cd983ba8d656fa457e3 => generated 297 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:02:36.517808 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c179e87-49b7-4e0d-8e22-674260c875d5 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:36.518095 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c179e87-49b7-4e0d-8e22-674260c875d5 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:36.518373 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c179e87-49b7-4e0d-8e22-674260c875d5 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:36.523344 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 122/251] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:02:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:02:36.525537 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2cbac7-5af1-477c-bcd5-7fbb95789688 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:36.525825 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2cbac7-5af1-477c-bcd5-7fbb95789688 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:36.526103 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2cbac7-5af1-477c-bcd5-7fbb95789688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:36.545684 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-ad2cbac7-5af1-477c-bcd5-7fbb95789688 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:36.546881 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-ad2cbac7-5af1-477c-bcd5-7fbb95789688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:36.556203 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154cb08b-3904-485f-a1b6-357577ab8842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:36.557201 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154cb08b-3904-485f-a1b6-357577ab8842 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:36.557546 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154cb08b-3904-485f-a1b6-357577ab8842 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:36.557891 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154cb08b-3904-485f-a1b6-357577ab8842 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:36.558788 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154cb08b-3904-485f-a1b6-357577ab8842 None admin] RBAC: Adding query filter params (role.id=5ab0887d24bc41b882f6786b7aab0fe8, scope.project.id=6c2de9abe1504f6698a9ea2a032c91e4, user.id=d360e0a89a50403aa353066d2b7fe68d) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:36.559158 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154cb08b-3904-485f-a1b6-357577ab8842 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:36.560794 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154cb08b-3904-485f-a1b6-357577ab8842 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:36.566775 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 130/252] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Aug 4 21:02:36 2022] GET /identity/v3/role_assignments?user.id=d360e0a89a50403aa353066d2b7fe68d&scope.project.id=6c2de9abe1504f6698a9ea2a032c91e4&role.id=5ab0887d24bc41b882f6786b7aab0fe8 => generated 259 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:36.567553 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154cb08b-3904-485f-a1b6-357577ab8842 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:36.684975 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 123/253] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:02:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 21:02:36.686469 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d11a59f-9e3e-4084-9a3b-0718d666152f None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:36.686736 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d11a59f-9e3e-4084-9a3b-0718d666152f None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:36.687010 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d11a59f-9e3e-4084-9a3b-0718d666152f None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:36.688507 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 124/254] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:02:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:02:36.694564 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7b85aa-e793-40b1-ad1c-005090d2590d None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:36.694862 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7b85aa-e793-40b1-ad1c-005090d2590d None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:36.695117 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7b85aa-e793-40b1-ad1c-005090d2590d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:36.726395 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154cb08b-3904-485f-a1b6-357577ab8842 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:36.729109 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-7a7b85aa-e793-40b1-ad1c-005090d2590d None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:36.730318 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-7a7b85aa-e793-40b1-ad1c-005090d2590d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:36.740684 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:36.788778 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0302291e-23f1-4e56-9ecf-ec4771bb2594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:36.789432 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0302291e-23f1-4e56-9ecf-ec4771bb2594 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:36.789719 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0302291e-23f1-4e56-9ecf-ec4771bb2594 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:36.789999 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0302291e-23f1-4e56-9ecf-ec4771bb2594 None admin] PATH_INFO: `/v3/projects/6c2de9abe1504f6698a9ea2a032c91e4` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:36.791709 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0302291e-23f1-4e56-9ecf-ec4771bb2594 None admin] RBAC: Authorizing `identity:get_project(project_id=6c2de9abe1504f6698a9ea2a032c91e4)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:36.793191 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0302291e-23f1-4e56-9ecf-ec4771bb2594 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:36.794897 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 131/255] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Aug 4 21:02:36 2022] GET /identity/v3/projects/6c2de9abe1504f6698a9ea2a032c91e4 => generated 302 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:36.795683 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0302291e-23f1-4e56-9ecf-ec4771bb2594 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:36.827693 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:36.855862 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 125/256] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:02:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 21:02:36.857038 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7b85aa-e793-40b1-ad1c-005090d2590d None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:36.955902 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0302291e-23f1-4e56-9ecf-ec4771bb2594 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:37.004592 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7b85aa-e793-40b1-ad1c-005090d2590d None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:37.013390 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a9099d-c577-4f6d-be78-4a0c13a54437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:37.014058 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a9099d-c577-4f6d-be78-4a0c13a54437 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:37.014341 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a9099d-c577-4f6d-be78-4a0c13a54437 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:37.014612 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a9099d-c577-4f6d-be78-4a0c13a54437 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:37.037109 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a9099d-c577-4f6d-be78-4a0c13a54437 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:02:37.044900 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-17a9099d-c577-4f6d-be78-4a0c13a54437 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:02:37.044900 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:02:37.044900 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:02:37.044900 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:02:37.044900 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:02:37.044900 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:02:37.044900 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:02:37.044900 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:02:37.044900 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:02:37.044900 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:02:37.044900 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:02:37.044900 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:02:37.044900 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:02:37.044900 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:02:37.044900 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:02:37.044900 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:02:37.045827 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:02:37.045827 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:02:37.045827 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:02:37.045827 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:02:37.045827 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:02:37.045827 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:02:37.045827 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:02:37.045827 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:02:37.045827 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:02:37.045827 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:02:37.045827 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:02:37.045827 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:02:37.045827 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:02:37.045827 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:02:37.045827 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:02:37.045827 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:02:37.046619 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:02:37.046619 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:02:37.046619 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:02:37.046619 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:02:37.046619 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:02:37.046619 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:02:37.046619 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:02:37.046619 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:02:37.046619 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:02:37.046619 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:02:37.046619 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:02:37.046619 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:02:37.046619 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 132/257] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Aug 4 21:02:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:02:37.046619 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a9099d-c577-4f6d-be78-4a0c13a54437 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:37.054751 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb71f1e8-7b88-427c-b564-aef3007e5677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:37.055589 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb71f1e8-7b88-427c-b564-aef3007e5677 None admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:37.056011 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb71f1e8-7b88-427c-b564-aef3007e5677 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:37.056390 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb71f1e8-7b88-427c-b564-aef3007e5677 None admin] PATH_INFO: `/v3/projects/6c2de9abe1504f6698a9ea2a032c91e4/users/c43671b71d544cd983ba8d656fa457e3/roles/8feebc60363c4feb870718b1721e5324` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:37.058546 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb71f1e8-7b88-427c-b564-aef3007e5677 None admin] RBAC: Authorizing `identity:create_grant(project_id=6c2de9abe1504f6698a9ea2a032c91e4, user_id=c43671b71d544cd983ba8d656fa457e3, role_id=8feebc60363c4feb870718b1721e5324)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:37.060167 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb71f1e8-7b88-427c-b564-aef3007e5677 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:37.097492 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:37.129875 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 126/258] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Aug 4 21:02:36 2022] PUT /identity/v3/projects/6c2de9abe1504f6698a9ea2a032c91e4/users/c43671b71d544cd983ba8d656fa457e3/roles/8feebc60363c4feb870718b1721e5324 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:02:37.130666 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb71f1e8-7b88-427c-b564-aef3007e5677 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:37.181091 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a9099d-c577-4f6d-be78-4a0c13a54437 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:37.225383 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009bf492-17d5-46c2-950f-0573224848e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:37.225970 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009bf492-17d5-46c2-950f-0573224848e2 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:37.226253 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009bf492-17d5-46c2-950f-0573224848e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:37.226532 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009bf492-17d5-46c2-950f-0573224848e2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:37.239053 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009bf492-17d5-46c2-950f-0573224848e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:02:37.239534 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:37.244964 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-009bf492-17d5-46c2-950f-0573224848e2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 21:02:37.244964 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:02:37.244964 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:02:37.244964 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:02:37.244964 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:02:37.244964 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:02:37.244964 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:02:37.244964 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:02:37.244964 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:02:37.244964 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:02:37.244964 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:02:37.244964 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:02:37.244964 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:02:37.244964 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:02:37.244964 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:02:37.244964 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:02:37.245900 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:02:37.245900 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:02:37.245900 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:02:37.245900 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:02:37.245900 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:02:37.245900 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:02:37.245900 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:02:37.245900 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:02:37.245900 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:02:37.245900 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:02:37.245900 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:02:37.245900 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:02:37.245900 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:02:37.245900 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:02:37.245900 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:02:37.245900 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:02:37.246925 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:02:37.246925 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:02:37.246925 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:02:37.246925 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:02:37.246925 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:02:37.246925 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:02:37.246925 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:02:37.246925 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:02:37.246925 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:02:37.246925 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:02:37.246925 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 21:02:37.246925 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:02:37.246925 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 133/259] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Aug 4 21:02:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:02:37.254259 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009bf492-17d5-46c2-950f-0573224848e2 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:37.420935 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb71f1e8-7b88-427c-b564-aef3007e5677 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:37.436435 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:37.444022 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009bf492-17d5-46c2-950f-0573224848e2 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:37.482246 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8988729a-eb73-4e8c-be30-7de8700c2414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:37.482864 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8988729a-eb73-4e8c-be30-7de8700c2414 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:37.483155 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8988729a-eb73-4e8c-be30-7de8700c2414 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:37.483480 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8988729a-eb73-4e8c-be30-7de8700c2414 None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:37.484216 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8988729a-eb73-4e8c-be30-7de8700c2414 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:37.484527 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8988729a-eb73-4e8c-be30-7de8700c2414 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:37.485994 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8988729a-eb73-4e8c-be30-7de8700c2414 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:37.495347 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 134/260] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Aug 4 21:02:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:37.500860 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8988729a-eb73-4e8c-be30-7de8700c2414 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:37.508053 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f01a7ae-da7c-458e-9601-895e624f1555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:37.508952 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f01a7ae-da7c-458e-9601-895e624f1555 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:37.509430 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f01a7ae-da7c-458e-9601-895e624f1555 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:37.509884 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f01a7ae-da7c-458e-9601-895e624f1555 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:37.510800 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f01a7ae-da7c-458e-9601-895e624f1555 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:37.511285 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f01a7ae-da7c-458e-9601-895e624f1555 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:37.512983 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f01a7ae-da7c-458e-9601-895e624f1555 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:37.529887 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 127/261] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Aug 4 21:02:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 399 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:37.550618 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f01a7ae-da7c-458e-9601-895e624f1555 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:37.611778 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8988729a-eb73-4e8c-be30-7de8700c2414 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:37.649053 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df3746f-5b7b-4388-b7ff-b1c6d91bc61d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:37.649713 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df3746f-5b7b-4388-b7ff-b1c6d91bc61d None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:37.650001 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df3746f-5b7b-4388-b7ff-b1c6d91bc61d None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:37.650278 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df3746f-5b7b-4388-b7ff-b1c6d91bc61d None admin] PATH_INFO: `/v3/users/93d8deefa5254382b257e6e3fbe3d481` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:37.651871 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df3746f-5b7b-4388-b7ff-b1c6d91bc61d None admin] RBAC: Authorizing `identity:get_user(user_id=93d8deefa5254382b257e6e3fbe3d481)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:37.653606 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df3746f-5b7b-4388-b7ff-b1c6d91bc61d None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:37.655397 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 135/262] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Aug 4 21:02:37 2022] GET /identity/v3/users/93d8deefa5254382b257e6e3fbe3d481 => generated 283 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:02:37.662865 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df3746f-5b7b-4388-b7ff-b1c6d91bc61d None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:37.695681 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:37.779573 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df3746f-5b7b-4388-b7ff-b1c6d91bc61d None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:37.809365 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f01a7ae-da7c-458e-9601-895e624f1555 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:37.820536 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a77d9f1-70e6-4bbd-bf14-121f37452450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:37.821311 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a77d9f1-70e6-4bbd-bf14-121f37452450 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:37.821674 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a77d9f1-70e6-4bbd-bf14-121f37452450 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:37.822026 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a77d9f1-70e6-4bbd-bf14-121f37452450 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:37.827656 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:37.883079 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c1c533-aa17-4402-af70-bb686b908cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:37.883747 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c1c533-aa17-4402-af70-bb686b908cf1 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:37.884032 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c1c533-aa17-4402-af70-bb686b908cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:37.884311 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c1c533-aa17-4402-af70-bb686b908cf1 None admin] PATH_INFO: `/v3/users/93d8deefa5254382b257e6e3fbe3d481` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:37.885473 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c1c533-aa17-4402-af70-bb686b908cf1 None admin] RBAC: Authorizing `identity:get_user(user_id=93d8deefa5254382b257e6e3fbe3d481)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:37.892150 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c1c533-aa17-4402-af70-bb686b908cf1 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:37.894644 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 128/263] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Aug 4 21:02:37 2022] GET /identity/v3/users/93d8deefa5254382b257e6e3fbe3d481 => generated 283 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:37.903992 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a77d9f1-70e6-4bbd-bf14-121f37452450 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:37.905468 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a77d9f1-70e6-4bbd-bf14-121f37452450 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:37.915145 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c1c533-aa17-4402-af70-bb686b908cf1 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:37.935575 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:38.012504 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 136/264] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Aug 4 21:02:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 21:02:38.018191 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a77d9f1-70e6-4bbd-bf14-121f37452450 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:38.081875 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c1c533-aa17-4402-af70-bb686b908cf1 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:38.121421 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a77d9f1-70e6-4bbd-bf14-121f37452450 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:38.140778 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac182b41-d516-4984-9278-091de17f5649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:38.141419 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac182b41-d516-4984-9278-091de17f5649 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:38.141699 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac182b41-d516-4984-9278-091de17f5649 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:38.141971 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac182b41-d516-4984-9278-091de17f5649 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:38.151971 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:38.157053 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a48ec1-5e2b-441f-989b-c75b1b1079bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:38.157712 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a48ec1-5e2b-441f-989b-c75b1b1079bc None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:38.157999 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a48ec1-5e2b-441f-989b-c75b1b1079bc None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:38.158539 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a48ec1-5e2b-441f-989b-c75b1b1079bc None admin] PATH_INFO: `/v3/projects/6c2de9abe1504f6698a9ea2a032c91e4` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:38.160858 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a48ec1-5e2b-441f-989b-c75b1b1079bc None admin] RBAC: Authorizing `identity:get_project(project_id=6c2de9abe1504f6698a9ea2a032c91e4)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:38.162323 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a48ec1-5e2b-441f-989b-c75b1b1079bc None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:38.168689 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 137/265] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Aug 4 21:02:38 2022] GET /identity/v3/projects/6c2de9abe1504f6698a9ea2a032c91e4 => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:02:38.176112 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a48ec1-5e2b-441f-989b-c75b1b1079bc None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:38.235660 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:38.278505 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac182b41-d516-4984-9278-091de17f5649 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:38.280611 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac182b41-d516-4984-9278-091de17f5649 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:38.388803 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a48ec1-5e2b-441f-989b-c75b1b1079bc None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:38.428735 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454c9eaa-f38a-4116-8c38-06b5ca2c6890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:38.429363 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454c9eaa-f38a-4116-8c38-06b5ca2c6890 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:38.429654 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454c9eaa-f38a-4116-8c38-06b5ca2c6890 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:38.429936 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454c9eaa-f38a-4116-8c38-06b5ca2c6890 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:38.430732 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454c9eaa-f38a-4116-8c38-06b5ca2c6890 None admin] RBAC: Adding query filter params (role.id=70587326d8584316b73b380df2ae8e67, scope.project.id=6c2de9abe1504f6698a9ea2a032c91e4, user.id=93d8deefa5254382b257e6e3fbe3d481) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:38.431042 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454c9eaa-f38a-4116-8c38-06b5ca2c6890 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:38.432537 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454c9eaa-f38a-4116-8c38-06b5ca2c6890 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:38.438896 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 138/266] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Aug 4 21:02:38 2022] GET /identity/v3/role_assignments?user.id=93d8deefa5254382b257e6e3fbe3d481&scope.project.id=6c2de9abe1504f6698a9ea2a032c91e4&role.id=70587326d8584316b73b380df2ae8e67 => generated 608 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:38.478478 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 129/267] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Aug 4 21:02:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 563 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 21:02:38.483750 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454c9eaa-f38a-4116-8c38-06b5ca2c6890 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:38.563046 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:38.622783 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454c9eaa-f38a-4116-8c38-06b5ca2c6890 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:38.661869 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d53478-70bc-4511-ba26-ce22a8db6a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:38.662537 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d53478-70bc-4511-ba26-ce22a8db6a4b None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:38.662822 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d53478-70bc-4511-ba26-ce22a8db6a4b None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:38.663098 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d53478-70bc-4511-ba26-ce22a8db6a4b None admin] PATH_INFO: `/v3/projects/6c2de9abe1504f6698a9ea2a032c91e4` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:38.664365 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d53478-70bc-4511-ba26-ce22a8db6a4b None admin] RBAC: Authorizing `identity:get_project(project_id=6c2de9abe1504f6698a9ea2a032c91e4)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:38.665861 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d53478-70bc-4511-ba26-ce22a8db6a4b None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:38.667986 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 139/268] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Aug 4 21:02:38 2022] GET /identity/v3/projects/6c2de9abe1504f6698a9ea2a032c91e4 => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:38.674237 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac182b41-d516-4984-9278-091de17f5649 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:38.722845 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b198e-fd94-462e-8071-3197931301e9 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:38.723142 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b198e-fd94-462e-8071-3197931301e9 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:38.723464 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b198e-fd94-462e-8071-3197931301e9 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:38.724903 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 140/269] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:02:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:02:38.731162 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fc8aff-66ad-4ee7-af2d-bb9d30df2085 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:38.731533 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fc8aff-66ad-4ee7-af2d-bb9d30df2085 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:38.731832 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fc8aff-66ad-4ee7-af2d-bb9d30df2085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:38.757985 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-15fc8aff-66ad-4ee7-af2d-bb9d30df2085 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:38.759439 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-15fc8aff-66ad-4ee7-af2d-bb9d30df2085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:38.800540 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:38.835723 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac182b41-d516-4984-9278-091de17f5649 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:38.854417 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 141/270] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:02:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:02:38.877227 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdd06fc-ed5a-4762-b671-d53cc3a06635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:38.877891 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdd06fc-ed5a-4762-b671-d53cc3a06635 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:38.878935 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdd06fc-ed5a-4762-b671-d53cc3a06635 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:38.879373 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdd06fc-ed5a-4762-b671-d53cc3a06635 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:38.880295 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdd06fc-ed5a-4762-b671-d53cc3a06635 None admin] RBAC: Adding query filter params (role.id=a432dd4735944ecfab2fa5ce1705d356, scope.project.id=6c2de9abe1504f6698a9ea2a032c91e4, user.id=93d8deefa5254382b257e6e3fbe3d481) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:38.880742 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdd06fc-ed5a-4762-b671-d53cc3a06635 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:38.882356 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdd06fc-ed5a-4762-b671-d53cc3a06635 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:38.888810 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 130/271] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Aug 4 21:02:38 2022] GET /identity/v3/role_assignments?user.id=93d8deefa5254382b257e6e3fbe3d481&scope.project.id=6c2de9abe1504f6698a9ea2a032c91e4&role.id=a432dd4735944ecfab2fa5ce1705d356 => generated 608 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:38.892858 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fc8aff-66ad-4ee7-af2d-bb9d30df2085 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:38.975372 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:39.027497 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fc8aff-66ad-4ee7-af2d-bb9d30df2085 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:39.030799 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d039728-6f4f-413a-8fd5-6ace3d3cbf35 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:39.031359 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d039728-6f4f-413a-8fd5-6ace3d3cbf35 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:39.031817 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d039728-6f4f-413a-8fd5-6ace3d3cbf35 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:39.033745 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 131/272] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:02:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:02:39.045330 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6877210e-8979-4638-83a3-75db8a6d67ce None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:39.045960 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6877210e-8979-4638-83a3-75db8a6d67ce None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:39.046438 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6877210e-8979-4638-83a3-75db8a6d67ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:39.068755 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-6877210e-8979-4638-83a3-75db8a6d67ce None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:39.070686 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-6877210e-8979-4638-83a3-75db8a6d67ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:39.087907 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57bfa14-e966-494e-8ba9-1b86a62fbfc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:39.088574 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57bfa14-e966-494e-8ba9-1b86a62fbfc7 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:39.088858 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57bfa14-e966-494e-8ba9-1b86a62fbfc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:39.089133 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57bfa14-e966-494e-8ba9-1b86a62fbfc7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:39.113307 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57bfa14-e966-494e-8ba9-1b86a62fbfc7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:02:39.118994 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-f57bfa14-e966-494e-8ba9-1b86a62fbfc7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 21:02:39.118994 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:02:39.118994 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:02:39.118994 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:02:39.118994 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:02:39.118994 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:02:39.118994 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:02:39.118994 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:02:39.118994 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:02:39.118994 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:02:39.118994 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:02:39.118994 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:02:39.118994 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:02:39.118994 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:02:39.118994 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:02:39.118994 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:02:39.119969 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:02:39.119969 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:02:39.119969 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:02:39.119969 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:02:39.119969 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:02:39.119969 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:02:39.119969 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:02:39.119969 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:02:39.119969 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:02:39.119969 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:02:39.119969 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:02:39.119969 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:02:39.119969 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:02:39.119969 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:02:39.119969 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:02:39.119969 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:02:39.120749 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:02:39.120749 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:02:39.120749 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:02:39.120749 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:02:39.120749 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:02:39.120749 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:02:39.120749 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:02:39.120749 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:02:39.120749 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:02:39.120749 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:02:39.120749 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 21:02:39.120749 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:02:39.123219 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 142/273] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Aug 4 21:02:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:02:39.130297 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57bfa14-e966-494e-8ba9-1b86a62fbfc7 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:39.211554 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 132/274] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:02:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:02:39.252449 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6877210e-8979-4638-83a3-75db8a6d67ce None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:39.261202 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57bfa14-e966-494e-8ba9-1b86a62fbfc7 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:39.287777 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:39.309630 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef157c7a-43dc-4455-8d7a-ca1c28e131f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:39.310643 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef157c7a-43dc-4455-8d7a-ca1c28e131f4 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:39.310931 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef157c7a-43dc-4455-8d7a-ca1c28e131f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:39.311209 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef157c7a-43dc-4455-8d7a-ca1c28e131f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:39.311994 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef157c7a-43dc-4455-8d7a-ca1c28e131f4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:39.312319 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef157c7a-43dc-4455-8d7a-ca1c28e131f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:39.314382 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef157c7a-43dc-4455-8d7a-ca1c28e131f4 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:39.327476 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 143/275] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Aug 4 21:02:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:39.333243 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef157c7a-43dc-4455-8d7a-ca1c28e131f4 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:39.359729 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:39.395552 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6877210e-8979-4638-83a3-75db8a6d67ce None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:39.434428 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dbb849-fd0d-4508-b9c6-e496b05fa93c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:39.435437 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dbb849-fd0d-4508-b9c6-e496b05fa93c None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:39.435778 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dbb849-fd0d-4508-b9c6-e496b05fa93c None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:39.436064 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dbb849-fd0d-4508-b9c6-e496b05fa93c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:39.438105 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dbb849-fd0d-4508-b9c6-e496b05fa93c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:39.440998 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dbb849-fd0d-4508-b9c6-e496b05fa93c None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:39.443389 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 133/276] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:02:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:02:39.448152 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dbb849-fd0d-4508-b9c6-e496b05fa93c None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:39.449864 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef157c7a-43dc-4455-8d7a-ca1c28e131f4 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:39.488358 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d1fd55-3a0b-4384-8cdf-d22eabab083c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:39.490192 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d1fd55-3a0b-4384-8cdf-d22eabab083c None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:39.490666 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d1fd55-3a0b-4384-8cdf-d22eabab083c None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:39.491117 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d1fd55-3a0b-4384-8cdf-d22eabab083c None admin] PATH_INFO: `/v3/users/0e48df4352b9423c8195eeb07f854ff5` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:39.507181 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d1fd55-3a0b-4384-8cdf-d22eabab083c None admin] RBAC: Authorizing `identity:get_user(user_id=0e48df4352b9423c8195eeb07f854ff5)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:39.508999 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d1fd55-3a0b-4384-8cdf-d22eabab083c None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:39.510903 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 144/277] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Aug 4 21:02:39 2022] GET /identity/v3/users/0e48df4352b9423c8195eeb07f854ff5 => generated 293 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:39.515304 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d772e7-d03d-4219-88fe-2d49263222c5 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:39.515696 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d772e7-d03d-4219-88fe-2d49263222c5 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:39.516100 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d772e7-d03d-4219-88fe-2d49263222c5 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:39.517481 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 145/278] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:02:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 21:02:39.520100 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d772e7-d03d-4219-88fe-2d49263222c5 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:39.562902 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:39.569235 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dbb849-fd0d-4508-b9c6-e496b05fa93c None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:39.604598 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d91c452-d98f-4ff4-b77f-7bf24c34a143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:39.605219 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d91c452-d98f-4ff4-b77f-7bf24c34a143 None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:39.605497 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d91c452-d98f-4ff4-b77f-7bf24c34a143 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:39.605888 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d91c452-d98f-4ff4-b77f-7bf24c34a143 None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:39.606563 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d91c452-d98f-4ff4-b77f-7bf24c34a143 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:39.608044 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d91c452-d98f-4ff4-b77f-7bf24c34a143 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:39.617540 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:39.631568 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d772e7-d03d-4219-88fe-2d49263222c5 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:39.668200 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5ee485-445f-44c6-a9f3-506daf22e283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:39.668831 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5ee485-445f-44c6-a9f3-506daf22e283 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:39.669112 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5ee485-445f-44c6-a9f3-506daf22e283 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:39.669395 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5ee485-445f-44c6-a9f3-506daf22e283 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:39.670187 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5ee485-445f-44c6-a9f3-506daf22e283 None admin] RBAC: Adding query filter params (role.id=8feebc60363c4feb870718b1721e5324, user.id=0e48df4352b9423c8195eeb07f854ff5, scope.system=all) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:39.670501 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5ee485-445f-44c6-a9f3-506daf22e283 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:39.672042 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5ee485-445f-44c6-a9f3-506daf22e283 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:39.677999 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 146/279] 10.222.0.25 () {56 vars in 1308 bytes} [Thu Aug 4 21:02:39 2022] GET /identity/v3/role_assignments?user.id=0e48df4352b9423c8195eeb07f854ff5&scope.system=all&role.id=8feebc60363c4feb870718b1721e5324 => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:39.680662 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3964aa7e-379f-417a-8030-c1dfbb76f61a None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:39.682908 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3964aa7e-379f-417a-8030-c1dfbb76f61a None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:39.682908 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3964aa7e-379f-417a-8030-c1dfbb76f61a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:39.700466 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-3964aa7e-379f-417a-8030-c1dfbb76f61a None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:39.701728 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-3964aa7e-379f-417a-8030-c1dfbb76f61a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:39.729454 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 134/280] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Aug 4 21:02:39 2022] POST /identity/v3/users => generated 293 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:02:39.799939 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 147/281] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:02:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:02:39.826893 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d91c452-d98f-4ff4-b77f-7bf24c34a143 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:39.942911 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d91c452-d98f-4ff4-b77f-7bf24c34a143 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:39.979180 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1079333-9f81-4895-acc9-7bdc212c3754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:39.979895 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1079333-9f81-4895-acc9-7bdc212c3754 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:39.980180 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1079333-9f81-4895-acc9-7bdc212c3754 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:39.980496 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1079333-9f81-4895-acc9-7bdc212c3754 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:39.993176 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1079333-9f81-4895-acc9-7bdc212c3754 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:02:39.999618 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-e1079333-9f81-4895-acc9-7bdc212c3754 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 21:02:39.999618 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:02:39.999618 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:02:39.999618 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:02:39.999618 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:02:39.999618 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:02:39.999618 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:02:39.999618 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:02:39.999618 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:02:39.999618 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:02:39.999618 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:02:39.999618 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:02:39.999618 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:02:39.999618 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:02:39.999618 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:02:39.999618 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:02:40.002391 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:02:40.002391 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:02:40.002391 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:02:40.002391 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:02:40.002391 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:02:40.002391 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:02:40.002391 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:02:40.002391 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:02:40.002391 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:02:40.002391 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:02:40.002391 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:02:40.002391 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:02:40.002391 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:02:40.002391 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:02:40.002391 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:02:40.002391 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:02:40.003239 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:02:40.003239 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:02:40.003239 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:02:40.003239 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:02:40.003239 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:02:40.003239 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:02:40.003239 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:02:40.003239 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:02:40.003239 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:02:40.003239 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:02:40.003239 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 21:02:40.003239 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:02:40.003239 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 135/282] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Aug 4 21:02:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:02:40.012560 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3964aa7e-379f-417a-8030-c1dfbb76f61a None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:40.056789 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:40.136115 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3964aa7e-379f-417a-8030-c1dfbb76f61a None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:40.173634 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e84a72-dbd5-4651-b9e2-5a01b84babdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:40.174639 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e84a72-dbd5-4651-b9e2-5a01b84babdd None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:40.175161 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e84a72-dbd5-4651-b9e2-5a01b84babdd None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:40.175727 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e84a72-dbd5-4651-b9e2-5a01b84babdd None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:40.176669 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e84a72-dbd5-4651-b9e2-5a01b84babdd None admin] RBAC: Adding query filter params (name=reader) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:40.177176 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e84a72-dbd5-4651-b9e2-5a01b84babdd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:40.178823 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e84a72-dbd5-4651-b9e2-5a01b84babdd None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:40.182756 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb75bf2b-4642-499b-a520-4a68434a91b4 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:40.183039 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb75bf2b-4642-499b-a520-4a68434a91b4 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:40.183341 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb75bf2b-4642-499b-a520-4a68434a91b4 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:40.184936 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 136/283] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:02:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 21:02:40.188683 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 148/284] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Aug 4 21:02:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:40.191247 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057325bc-82ac-492c-b854-95902583a3f2 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:40.191581 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057325bc-82ac-492c-b854-95902583a3f2 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:40.191863 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057325bc-82ac-492c-b854-95902583a3f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:40.201095 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e84a72-dbd5-4651-b9e2-5a01b84babdd None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:40.214344 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-057325bc-82ac-492c-b854-95902583a3f2 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:40.216451 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-057325bc-82ac-492c-b854-95902583a3f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:40.294946 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:40.307023 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:40.317245 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e84a72-dbd5-4651-b9e2-5a01b84babdd None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:40.321066 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 137/285] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:02:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:02:40.351229 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057325bc-82ac-492c-b854-95902583a3f2 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:40.375336 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54ae65c-513e-4dc4-9021-a0fd0fd42715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:40.376253 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ae65c-513e-4dc4-9021-a0fd0fd42715 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:40.376686 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ae65c-513e-4dc4-9021-a0fd0fd42715 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:40.377140 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ae65c-513e-4dc4-9021-a0fd0fd42715 None admin] PATH_INFO: `/v3/users/d360e0a89a50403aa353066d2b7fe68d` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:40.379172 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54ae65c-513e-4dc4-9021-a0fd0fd42715 None admin] RBAC: Authorizing `identity:get_user(user_id=d360e0a89a50403aa353066d2b7fe68d)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:40.381080 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54ae65c-513e-4dc4-9021-a0fd0fd42715 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:40.386188 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 149/286] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Aug 4 21:02:40 2022] GET /identity/v3/users/d360e0a89a50403aa353066d2b7fe68d => generated 297 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:02:40.390494 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54ae65c-513e-4dc4-9021-a0fd0fd42715 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:40.527505 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54ae65c-513e-4dc4-9021-a0fd0fd42715 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:40.564413 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e52e56-ef35-4c0b-8ba1-02569f82f199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:40.565293 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e52e56-ef35-4c0b-8ba1-02569f82f199 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:40.566257 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e52e56-ef35-4c0b-8ba1-02569f82f199 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:40.566677 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e52e56-ef35-4c0b-8ba1-02569f82f199 None admin] PATH_INFO: `/v3/projects/6c2de9abe1504f6698a9ea2a032c91e4` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:40.568030 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e52e56-ef35-4c0b-8ba1-02569f82f199 None admin] RBAC: Authorizing `identity:get_project(project_id=6c2de9abe1504f6698a9ea2a032c91e4)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:40.569672 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e52e56-ef35-4c0b-8ba1-02569f82f199 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:40.571522 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 150/287] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Aug 4 21:02:40 2022] GET /identity/v3/projects/6c2de9abe1504f6698a9ea2a032c91e4 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:40.573622 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057325bc-82ac-492c-b854-95902583a3f2 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:40.580494 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e52e56-ef35-4c0b-8ba1-02569f82f199 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:40.622993 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:40.676010 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cefd45-2ae1-40a8-9925-f3d9b651aac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:40.676670 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cefd45-2ae1-40a8-9925-f3d9b651aac4 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:40.676955 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cefd45-2ae1-40a8-9925-f3d9b651aac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:40.677230 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cefd45-2ae1-40a8-9925-f3d9b651aac4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:40.704262 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cefd45-2ae1-40a8-9925-f3d9b651aac4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:02:40.709477 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-21cefd45-2ae1-40a8-9925-f3d9b651aac4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 21:02:40.709477 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:02:40.709477 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:02:40.709477 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:02:40.709477 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:02:40.709477 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:02:40.709477 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:02:40.709477 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:02:40.709477 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:02:40.709477 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:02:40.709477 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:02:40.709477 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:02:40.709477 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:02:40.709477 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:02:40.709477 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:02:40.709477 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:02:40.710361 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:02:40.710361 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:02:40.710361 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:02:40.710361 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:02:40.710361 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:02:40.710361 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:02:40.710361 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:02:40.710361 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:02:40.710361 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:02:40.710361 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:02:40.710361 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:02:40.710361 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:02:40.710361 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:02:40.710361 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:02:40.710361 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:02:40.710361 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:02:40.711162 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:02:40.711162 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:02:40.711162 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:02:40.711162 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:02:40.711162 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:02:40.711162 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:02:40.711162 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:02:40.711162 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:02:40.711162 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:02:40.711162 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:02:40.711162 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 21:02:40.711162 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:02:40.719762 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 138/288] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Aug 4 21:02:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 369 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:02:40.720175 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e52e56-ef35-4c0b-8ba1-02569f82f199 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:40.726145 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cefd45-2ae1-40a8-9925-f3d9b651aac4 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:40.761458 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa02d81-b65f-40bd-976c-20b9a213ac93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:40.762570 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa02d81-b65f-40bd-976c-20b9a213ac93 None admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:40.763036 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa02d81-b65f-40bd-976c-20b9a213ac93 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:40.763512 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa02d81-b65f-40bd-976c-20b9a213ac93 None admin] PATH_INFO: `/v3/projects/6c2de9abe1504f6698a9ea2a032c91e4/users/d360e0a89a50403aa353066d2b7fe68d/roles/5ab0887d24bc41b882f6786b7aab0fe8` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:40.765756 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa02d81-b65f-40bd-976c-20b9a213ac93 None admin] RBAC: Authorizing `identity:create_grant(project_id=6c2de9abe1504f6698a9ea2a032c91e4, user_id=d360e0a89a50403aa353066d2b7fe68d, role_id=5ab0887d24bc41b882f6786b7aab0fe8)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:40.767422 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa02d81-b65f-40bd-976c-20b9a213ac93 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:40.809415 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:40.848068 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 151/289] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Aug 4 21:02:40 2022] PUT /identity/v3/projects/6c2de9abe1504f6698a9ea2a032c91e4/users/d360e0a89a50403aa353066d2b7fe68d/roles/5ab0887d24bc41b882f6786b7aab0fe8 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:02:40.890548 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cefd45-2ae1-40a8-9925-f3d9b651aac4 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:40.941927 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952aac61-2cec-4dd0-818e-d36057b21082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:40.942941 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952aac61-2cec-4dd0-818e-d36057b21082 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:40.943857 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952aac61-2cec-4dd0-818e-d36057b21082 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:40.944313 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952aac61-2cec-4dd0-818e-d36057b21082 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:40.945293 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952aac61-2cec-4dd0-818e-d36057b21082 None admin] RBAC: Adding query filter params (name=member) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:40.945794 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952aac61-2cec-4dd0-818e-d36057b21082 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:40.947417 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952aac61-2cec-4dd0-818e-d36057b21082 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:40.957405 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 139/290] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Aug 4 21:02:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:02:40.963369 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa02d81-b65f-40bd-976c-20b9a213ac93 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:41.087790 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:41.100854 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa02d81-b65f-40bd-976c-20b9a213ac93 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:41.142685 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f91f6c-4fbb-4968-86e0-0d8252f04fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:41.144352 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f91f6c-4fbb-4968-86e0-0d8252f04fe6 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:41.144992 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f91f6c-4fbb-4968-86e0-0d8252f04fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:41.145272 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f91f6c-4fbb-4968-86e0-0d8252f04fe6 None admin] PATH_INFO: `/v3/users/c43671b71d544cd983ba8d656fa457e3` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:41.146860 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f91f6c-4fbb-4968-86e0-0d8252f04fe6 None admin] RBAC: Authorizing `identity:get_user(user_id=c43671b71d544cd983ba8d656fa457e3)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:41.148344 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f91f6c-4fbb-4968-86e0-0d8252f04fe6 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:41.150371 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 152/291] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Aug 4 21:02:40 2022] GET /identity/v3/users/c43671b71d544cd983ba8d656fa457e3 => generated 297 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:41.155965 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952aac61-2cec-4dd0-818e-d36057b21082 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:41.303617 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952aac61-2cec-4dd0-818e-d36057b21082 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:41.309103 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:41.353630 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9781d38d-7bc2-4859-8247-7cd73eb3c416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:41.354193 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9781d38d-7bc2-4859-8247-7cd73eb3c416 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:41.354475 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9781d38d-7bc2-4859-8247-7cd73eb3c416 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:41.354757 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9781d38d-7bc2-4859-8247-7cd73eb3c416 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:41.457136 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9781d38d-7bc2-4859-8247-7cd73eb3c416 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:41.458639 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9781d38d-7bc2-4859-8247-7cd73eb3c416 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:41.592969 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 140/292] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Aug 4 21:02:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 21:02:41.598976 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f91f6c-4fbb-4968-86e0-0d8252f04fe6 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:41.742953 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f91f6c-4fbb-4968-86e0-0d8252f04fe6 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:41.792152 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd1b54c-fe91-4983-9298-39b7a6dfff19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:41.792803 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd1b54c-fe91-4983-9298-39b7a6dfff19 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:41.793091 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd1b54c-fe91-4983-9298-39b7a6dfff19 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:41.793367 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd1b54c-fe91-4983-9298-39b7a6dfff19 None admin] PATH_INFO: `/v3/projects/6c2de9abe1504f6698a9ea2a032c91e4` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:41.798477 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd1b54c-fe91-4983-9298-39b7a6dfff19 None admin] RBAC: Authorizing `identity:get_project(project_id=6c2de9abe1504f6698a9ea2a032c91e4)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:41.800001 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd1b54c-fe91-4983-9298-39b7a6dfff19 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:41.801842 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 153/293] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Aug 4 21:02:41 2022] GET /identity/v3/projects/6c2de9abe1504f6698a9ea2a032c91e4 => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:41.807966 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9781d38d-7bc2-4859-8247-7cd73eb3c416 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:41.816452 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:41.922971 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d8636f-e245-420a-8145-0636f10c1750 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:41.923159 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d8636f-e245-420a-8145-0636f10c1750 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:41.923440 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d8636f-e245-420a-8145-0636f10c1750 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:41.925572 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 154/294] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:02:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:02:41.940583 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e793343-7ad3-499b-bf7d-7d292578b96b None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:41.940939 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e793343-7ad3-499b-bf7d-7d292578b96b None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:41.941222 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e793343-7ad3-499b-bf7d-7d292578b96b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:41.950490 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9781d38d-7bc2-4859-8247-7cd73eb3c416 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:41.991787 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-5e793343-7ad3-499b-bf7d-7d292578b96b None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:41.997671 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-5e793343-7ad3-499b-bf7d-7d292578b96b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:41.998162 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e683f9-0fb3-4817-befe-2a3567eb7fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:41.999036 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e683f9-0fb3-4817-befe-2a3567eb7fd8 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:41.999863 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e683f9-0fb3-4817-befe-2a3567eb7fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:42.000637 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e683f9-0fb3-4817-befe-2a3567eb7fd8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:42.002295 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e683f9-0fb3-4817-befe-2a3567eb7fd8 None admin] RBAC: Adding query filter params (role.id=8feebc60363c4feb870718b1721e5324, scope.project.id=6c2de9abe1504f6698a9ea2a032c91e4, user.id=c43671b71d544cd983ba8d656fa457e3) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:42.003666 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e683f9-0fb3-4817-befe-2a3567eb7fd8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:42.005558 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e683f9-0fb3-4817-befe-2a3567eb7fd8 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:42.013955 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 141/295] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Aug 4 21:02:41 2022] GET /identity/v3/role_assignments?user.id=c43671b71d544cd983ba8d656fa457e3&scope.project.id=6c2de9abe1504f6698a9ea2a032c91e4&role.id=8feebc60363c4feb870718b1721e5324 => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:42.023218 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:42.081967 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 155/296] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:02:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 21:02:42.120881 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e683f9-0fb3-4817-befe-2a3567eb7fd8 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:42.203210 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:42.247143 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e683f9-0fb3-4817-befe-2a3567eb7fd8 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:42.305511 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275609cc-305b-4331-bca2-6ffa48775b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:42.305892 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275609cc-305b-4331-bca2-6ffa48775b8f None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:42.306182 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275609cc-305b-4331-bca2-6ffa48775b8f None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:42.306459 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275609cc-305b-4331-bca2-6ffa48775b8f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:42.318153 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275609cc-305b-4331-bca2-6ffa48775b8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:02:42.323724 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-275609cc-305b-4331-bca2-6ffa48775b8f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 21:02:42.323724 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:02:42.323724 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:02:42.323724 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:02:42.323724 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:02:42.323724 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:02:42.323724 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:02:42.323724 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:02:42.323724 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:02:42.323724 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:02:42.323724 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:02:42.323724 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:02:42.323724 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:02:42.323724 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:02:42.323724 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:02:42.323724 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:02:42.324591 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:02:42.324591 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:02:42.324591 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:02:42.324591 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:02:42.324591 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:02:42.324591 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:02:42.324591 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:02:42.324591 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:02:42.324591 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:02:42.324591 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:02:42.324591 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:02:42.324591 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:02:42.324591 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:02:42.324591 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:02:42.324591 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:02:42.324591 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:02:42.325407 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:02:42.325407 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:02:42.325407 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:02:42.325407 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:02:42.325407 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:02:42.325407 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:02:42.325407 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:02:42.325407 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:02:42.325407 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:02:42.325407 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:02:42.325407 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 21:02:42.325407 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:02:42.329797 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 142/297] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Aug 4 21:02:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:02:42.331253 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e793343-7ad3-499b-bf7d-7d292578b96b None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:42.449906 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e793343-7ad3-499b-bf7d-7d292578b96b None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:42.489453 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b650391-dbeb-4997-b891-7c8d4ed67768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:42.490146 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b650391-dbeb-4997-b891-7c8d4ed67768 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:42.490435 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b650391-dbeb-4997-b891-7c8d4ed67768 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:42.490710 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b650391-dbeb-4997-b891-7c8d4ed67768 None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:42.491485 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b650391-dbeb-4997-b891-7c8d4ed67768 None admin] RBAC: Adding query filter params (name=member) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:42.491799 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b650391-dbeb-4997-b891-7c8d4ed67768 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:42.493185 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b650391-dbeb-4997-b891-7c8d4ed67768 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:42.506899 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 156/298] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Aug 4 21:02:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:42.508283 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275609cc-305b-4331-bca2-6ffa48775b8f None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:42.555419 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a49f159-d2da-456c-a286-8fb818d022c7 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:42.556124 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a49f159-d2da-456c-a286-8fb818d022c7 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:42.556351 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a49f159-d2da-456c-a286-8fb818d022c7 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:42.558425 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 157/299] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:02:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:02:42.564606 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3ab217-a444-4028-88a0-2b5339fc83e9 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:42.564896 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3ab217-a444-4028-88a0-2b5339fc83e9 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:42.565177 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3ab217-a444-4028-88a0-2b5339fc83e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:42.591067 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-5b3ab217-a444-4028-88a0-2b5339fc83e9 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:42.592354 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-5b3ab217-a444-4028-88a0-2b5339fc83e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:42.606545 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:42.633391 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275609cc-305b-4331-bca2-6ffa48775b8f None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:42.669478 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:42.678715 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2236b3b-d6e9-4a73-8160-4ecb6493c34b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:42.679425 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2236b3b-d6e9-4a73-8160-4ecb6493c34b None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:42.679703 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2236b3b-d6e9-4a73-8160-4ecb6493c34b None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:42.679983 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2236b3b-d6e9-4a73-8160-4ecb6493c34b None admin] PATH_INFO: `/v3/users/0e48df4352b9423c8195eeb07f854ff5` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:42.683678 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2236b3b-d6e9-4a73-8160-4ecb6493c34b None admin] RBAC: Authorizing `identity:get_user(user_id=0e48df4352b9423c8195eeb07f854ff5)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:42.683678 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2236b3b-d6e9-4a73-8160-4ecb6493c34b None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:42.684381 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 143/300] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Aug 4 21:02:42 2022] GET /identity/v3/users/0e48df4352b9423c8195eeb07f854ff5 => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:42.685433 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b6cab-10d0-4530-a04f-fddbcaa04506 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:42.686531 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b6cab-10d0-4530-a04f-fddbcaa04506 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:42.686531 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b6cab-10d0-4530-a04f-fddbcaa04506 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:42.686712 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 158/301] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:02:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:02:42.690966 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 144/302] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:02:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:02:42.691089 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3ab217-a444-4028-88a0-2b5339fc83e9 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:42.696441 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d768a-59f9-4b5a-ad4c-541570023526 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:42.696847 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d768a-59f9-4b5a-ad4c-541570023526 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:42.697236 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d768a-59f9-4b5a-ad4c-541570023526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:42.716318 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-e94d768a-59f9-4b5a-ad4c-541570023526 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:42.717524 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-e94d768a-59f9-4b5a-ad4c-541570023526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:42.785431 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 145/303] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:02:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 21:02:42.788328 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94d768a-59f9-4b5a-ad4c-541570023526 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:42.804179 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3ab217-a444-4028-88a0-2b5339fc83e9 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:42.839579 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915d5846-559c-4a4d-b016-ac9cc6e06357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:42.840158 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d5846-559c-4a4d-b016-ac9cc6e06357 None admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:42.840439 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d5846-559c-4a4d-b016-ac9cc6e06357 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:42.840719 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d5846-559c-4a4d-b016-ac9cc6e06357 None admin] PATH_INFO: `/v3/system/users/0e48df4352b9423c8195eeb07f854ff5/roles/8feebc60363c4feb870718b1721e5324` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:42.842282 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915d5846-559c-4a4d-b016-ac9cc6e06357 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=0e48df4352b9423c8195eeb07f854ff5, role_id=8feebc60363c4feb870718b1721e5324)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:42.843691 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915d5846-559c-4a4d-b016-ac9cc6e06357 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:42.894714 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94d768a-59f9-4b5a-ad4c-541570023526 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:42.920647 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:42.931071 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabc4ac6-0fd4-4044-a0d2-52f26ede0eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:42.931758 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabc4ac6-0fd4-4044-a0d2-52f26ede0eca None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:42.932038 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabc4ac6-0fd4-4044-a0d2-52f26ede0eca None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:42.932313 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabc4ac6-0fd4-4044-a0d2-52f26ede0eca None admin] PATH_INFO: `/v3/domains/default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:42.933442 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabc4ac6-0fd4-4044-a0d2-52f26ede0eca None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:42.934899 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabc4ac6-0fd4-4044-a0d2-52f26ede0eca None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:42.936564 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 146/304] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:02:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:42.938905 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 159/305] 10.222.0.25 () {58 vars in 1334 bytes} [Thu Aug 4 21:02:42 2022] PUT /identity/v3/system/users/0e48df4352b9423c8195eeb07f854ff5/roles/8feebc60363c4feb870718b1721e5324 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:02:42.942104 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabc4ac6-0fd4-4044-a0d2-52f26ede0eca None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:42.948515 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915d5846-559c-4a4d-b016-ac9cc6e06357 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:42.973180 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:43.093089 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabc4ac6-0fd4-4044-a0d2-52f26ede0eca None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:43.100401 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915d5846-559c-4a4d-b016-ac9cc6e06357 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:43.164915 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd59e0cd-1455-4c3c-8cf1-3520336e22e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:43.165555 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd59e0cd-1455-4c3c-8cf1-3520336e22e0 None admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:43.165837 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd59e0cd-1455-4c3c-8cf1-3520336e22e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:43.166070 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608bec77-3faf-40c1-ae85-a6b3b990349b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:43.166196 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd59e0cd-1455-4c3c-8cf1-3520336e22e0 None admin] PATH_INFO: `/v3/groups` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:43.166773 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd59e0cd-1455-4c3c-8cf1-3520336e22e0 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:43.166873 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608bec77-3faf-40c1-ae85-a6b3b990349b None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:43.167175 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608bec77-3faf-40c1-ae85-a6b3b990349b None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:43.167538 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608bec77-3faf-40c1-ae85-a6b3b990349b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:43.168254 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd59e0cd-1455-4c3c-8cf1-3520336e22e0 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:43.178528 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608bec77-3faf-40c1-ae85-a6b3b990349b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:02:43.184152 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-608bec77-3faf-40c1-ae85-a6b3b990349b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 21:02:43.184152 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:02:43.184152 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:02:43.184152 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:02:43.184152 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:02:43.184152 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:02:43.184152 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:02:43.184152 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:02:43.184152 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:02:43.184152 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:02:43.184152 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:02:43.184152 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:02:43.184152 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:02:43.184152 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:02:43.184152 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:02:43.184152 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:02:43.185027 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:02:43.185027 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:02:43.185027 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:02:43.185027 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:02:43.185027 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:02:43.185027 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:02:43.185027 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:02:43.185027 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:02:43.185027 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:02:43.185027 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:02:43.185027 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:02:43.185027 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:02:43.185027 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:02:43.185027 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:02:43.185027 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:02:43.185027 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:02:43.186074 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:02:43.186074 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:02:43.186074 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:02:43.186074 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:02:43.186074 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:02:43.186074 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:02:43.186074 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:02:43.186074 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:02:43.186074 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:02:43.186074 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:02:43.186074 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 21:02:43.186074 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:02:43.186074 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 147/306] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Aug 4 21:02:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:02:43.191094 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608bec77-3faf-40c1-ae85-a6b3b990349b None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:43.240903 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 160/307] 10.222.0.25 () {60 vars in 1134 bytes} [Thu Aug 4 21:02:42 2022] POST /identity/v3/groups => generated 229 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:02:43.266085 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:43.296554 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a3f7eb-1d82-41a0-931a-27d67af4aed7 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:43.297199 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a3f7eb-1d82-41a0-931a-27d67af4aed7 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:43.297683 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a3f7eb-1d82-41a0-931a-27d67af4aed7 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:43.299374 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 161/308] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:02:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 21:02:43.309024 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f0d69d-2f17-48cc-9fe7-ad3148efe7d1 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:43.309024 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f0d69d-2f17-48cc-9fe7-ad3148efe7d1 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:43.309024 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f0d69d-2f17-48cc-9fe7-ad3148efe7d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:43.319194 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608bec77-3faf-40c1-ae85-a6b3b990349b None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:43.328674 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-01f0d69d-2f17-48cc-9fe7-ad3148efe7d1 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:43.330463 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-01f0d69d-2f17-48cc-9fe7-ad3148efe7d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:43.361683 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282049bd-6df2-4aac-b447-77c8de3e56e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:43.362793 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282049bd-6df2-4aac-b447-77c8de3e56e4 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:43.363082 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282049bd-6df2-4aac-b447-77c8de3e56e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:43.363384 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282049bd-6df2-4aac-b447-77c8de3e56e4 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:43.364131 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282049bd-6df2-4aac-b447-77c8de3e56e4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:43.364437 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282049bd-6df2-4aac-b447-77c8de3e56e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:43.366259 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282049bd-6df2-4aac-b447-77c8de3e56e4 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:43.380612 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 148/309] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Aug 4 21:02:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:02:43.382340 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282049bd-6df2-4aac-b447-77c8de3e56e4 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:43.419523 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:43.425405 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 162/310] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:02:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:02:43.449265 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f0d69d-2f17-48cc-9fe7-ad3148efe7d1 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:43.517127 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282049bd-6df2-4aac-b447-77c8de3e56e4 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:43.540587 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:43.559184 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f0d69d-2f17-48cc-9fe7-ad3148efe7d1 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:43.564939 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbce8bc6-72b0-4198-b0b8-0b554332bf57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:43.566141 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbce8bc6-72b0-4198-b0b8-0b554332bf57 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:43.567130 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbce8bc6-72b0-4198-b0b8-0b554332bf57 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:43.567445 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbce8bc6-72b0-4198-b0b8-0b554332bf57 None admin] PATH_INFO: `/v3/users/bb0b5e8554204087a31b901e2195eb1f` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:43.585909 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbce8bc6-72b0-4198-b0b8-0b554332bf57 None admin] RBAC: Authorizing `identity:get_user(user_id=bb0b5e8554204087a31b901e2195eb1f)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:43.587938 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbce8bc6-72b0-4198-b0b8-0b554332bf57 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:43.591359 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 149/311] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Aug 4 21:02:43 2022] GET /identity/v3/users/bb0b5e8554204087a31b901e2195eb1f => generated 293 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:43.596148 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbce8bc6-72b0-4198-b0b8-0b554332bf57 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:43.609975 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740ca6e8-332d-4c08-b9ce-962c31b534ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:43.611270 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740ca6e8-332d-4c08-b9ce-962c31b534ac None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:43.612334 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740ca6e8-332d-4c08-b9ce-962c31b534ac None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:43.612830 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740ca6e8-332d-4c08-b9ce-962c31b534ac None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:43.628758 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740ca6e8-332d-4c08-b9ce-962c31b534ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:02:43.635612 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-740ca6e8-332d-4c08-b9ce-962c31b534ac None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 21:02:43.635612 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:02:43.635612 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:02:43.635612 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:02:43.635612 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:02:43.635612 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:02:43.635612 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:02:43.635612 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:02:43.635612 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:02:43.635612 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:02:43.635612 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:02:43.635612 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:02:43.635612 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:02:43.635612 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:02:43.635612 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:02:43.635612 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:02:43.636483 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:02:43.636483 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:02:43.636483 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:02:43.636483 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:02:43.636483 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:02:43.636483 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:02:43.636483 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:02:43.636483 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:02:43.636483 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:02:43.636483 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:02:43.636483 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:02:43.636483 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:02:43.636483 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:02:43.636483 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:02:43.636483 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:02:43.636483 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:02:43.637246 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:02:43.637246 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:02:43.637246 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:02:43.637246 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:02:43.637246 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:02:43.637246 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:02:43.637246 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:02:43.637246 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:02:43.637246 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:02:43.637246 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:02:43.637246 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 21:02:43.637246 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:02:43.640446 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 163/312] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Aug 4 21:02:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:02:43.646258 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740ca6e8-332d-4c08-b9ce-962c31b534ac None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:43.710692 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbce8bc6-72b0-4198-b0b8-0b554332bf57 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:43.743763 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:43.749413 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a4a2be-b3ab-4c31-ba20-22097008f0b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:43.750046 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a4a2be-b3ab-4c31-ba20-22097008f0b6 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:43.750329 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a4a2be-b3ab-4c31-ba20-22097008f0b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:43.750605 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a4a2be-b3ab-4c31-ba20-22097008f0b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:43.751431 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a4a2be-b3ab-4c31-ba20-22097008f0b6 None admin] RBAC: Adding query filter params (role.id=5ab0887d24bc41b882f6786b7aab0fe8, user.id=bb0b5e8554204087a31b901e2195eb1f, scope.system=all) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:43.751751 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a4a2be-b3ab-4c31-ba20-22097008f0b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:43.753745 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a4a2be-b3ab-4c31-ba20-22097008f0b6 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:43.759157 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 150/313] 10.222.0.25 () {56 vars in 1308 bytes} [Thu Aug 4 21:02:43 2022] GET /identity/v3/role_assignments?user.id=bb0b5e8554204087a31b901e2195eb1f&scope.system=all&role.id=5ab0887d24bc41b882f6786b7aab0fe8 => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:02:43.775626 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740ca6e8-332d-4c08-b9ce-962c31b534ac None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:43.816672 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8ff8e5-988e-433b-a344-70589fa8e8d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:43.817327 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8ff8e5-988e-433b-a344-70589fa8e8d5 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:43.817608 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8ff8e5-988e-433b-a344-70589fa8e8d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:43.817877 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8ff8e5-988e-433b-a344-70589fa8e8d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:43.818612 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8ff8e5-988e-433b-a344-70589fa8e8d5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:43.818913 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8ff8e5-988e-433b-a344-70589fa8e8d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:43.820373 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8ff8e5-988e-433b-a344-70589fa8e8d5 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:43.830451 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 164/314] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Aug 4 21:02:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:02:43.836118 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a4a2be-b3ab-4c31-ba20-22097008f0b6 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:43.882461 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:43.948360 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a4a2be-b3ab-4c31-ba20-22097008f0b6 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:43.984023 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33906a81-a1ca-4daf-b2dd-d024fc09bbd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:43.984673 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33906a81-a1ca-4daf-b2dd-d024fc09bbd6 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:43.984959 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33906a81-a1ca-4daf-b2dd-d024fc09bbd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:43.985237 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33906a81-a1ca-4daf-b2dd-d024fc09bbd6 None admin] PATH_INFO: `/v3/users/d360e0a89a50403aa353066d2b7fe68d` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:43.986406 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33906a81-a1ca-4daf-b2dd-d024fc09bbd6 None admin] RBAC: Authorizing `identity:get_user(user_id=d360e0a89a50403aa353066d2b7fe68d)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:43.988113 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33906a81-a1ca-4daf-b2dd-d024fc09bbd6 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:43.990031 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 151/315] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Aug 4 21:02:43 2022] GET /identity/v3/users/d360e0a89a50403aa353066d2b7fe68d => generated 297 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:43.995541 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8ff8e5-988e-433b-a344-70589fa8e8d5 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:44.188554 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8ff8e5-988e-433b-a344-70589fa8e8d5 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:44.215670 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:44.223925 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7640c0d-405e-44f1-ac61-b2d62e71c158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:44.224961 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7640c0d-405e-44f1-ac61-b2d62e71c158 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:44.224961 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7640c0d-405e-44f1-ac61-b2d62e71c158 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:44.229493 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7640c0d-405e-44f1-ac61-b2d62e71c158 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:44.296337 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7640c0d-405e-44f1-ac61-b2d62e71c158 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:44.297880 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7640c0d-405e-44f1-ac61-b2d62e71c158 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:44.401119 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 165/316] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Aug 4 21:02:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 21:02:44.406732 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33906a81-a1ca-4daf-b2dd-d024fc09bbd6 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:44.513177 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33906a81-a1ca-4daf-b2dd-d024fc09bbd6 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:44.520470 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:44.548551 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c6185b-c515-4d09-b656-37ff18827e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:44.549203 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c6185b-c515-4d09-b656-37ff18827e50 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:44.549494 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c6185b-c515-4d09-b656-37ff18827e50 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:44.549834 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c6185b-c515-4d09-b656-37ff18827e50 None admin] PATH_INFO: `/v3/projects/6c2de9abe1504f6698a9ea2a032c91e4` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:44.551007 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c6185b-c515-4d09-b656-37ff18827e50 None admin] RBAC: Authorizing `identity:get_project(project_id=6c2de9abe1504f6698a9ea2a032c91e4)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:44.552506 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c6185b-c515-4d09-b656-37ff18827e50 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:44.554482 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 152/317] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Aug 4 21:02:44 2022] GET /identity/v3/projects/6c2de9abe1504f6698a9ea2a032c91e4 => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:44.560089 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7640c0d-405e-44f1-ac61-b2d62e71c158 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:44.586652 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:44.669612 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7640c0d-405e-44f1-ac61-b2d62e71c158 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:44.704911 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9038ed6b-512c-40eb-8599-5949c5f62d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:44.705530 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9038ed6b-512c-40eb-8599-5949c5f62d26 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:44.705813 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9038ed6b-512c-40eb-8599-5949c5f62d26 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:44.706234 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9038ed6b-512c-40eb-8599-5949c5f62d26 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:44.707022 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9038ed6b-512c-40eb-8599-5949c5f62d26 None admin] RBAC: Adding query filter params (role.id=5ab0887d24bc41b882f6786b7aab0fe8, scope.project.id=6c2de9abe1504f6698a9ea2a032c91e4, user.id=d360e0a89a50403aa353066d2b7fe68d) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:44.707344 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9038ed6b-512c-40eb-8599-5949c5f62d26 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:44.708860 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9038ed6b-512c-40eb-8599-5949c5f62d26 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:44.715136 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 166/318] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Aug 4 21:02:44 2022] GET /identity/v3/role_assignments?user.id=d360e0a89a50403aa353066d2b7fe68d&scope.project.id=6c2de9abe1504f6698a9ea2a032c91e4&role.id=5ab0887d24bc41b882f6786b7aab0fe8 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:45.076024 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3434fa42-e131-4029-9543-21ddfa7cecda None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:45.076401 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3434fa42-e131-4029-9543-21ddfa7cecda None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:45.076567 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3434fa42-e131-4029-9543-21ddfa7cecda None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:45.078425 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 153/319] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:02:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:02:45.083363 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44869f7-f21b-4991-89be-b594d823843c None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:45.083659 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44869f7-f21b-4991-89be-b594d823843c None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:45.083935 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44869f7-f21b-4991-89be-b594d823843c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:45.102205 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-a44869f7-f21b-4991-89be-b594d823843c None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:45.103467 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-a44869f7-f21b-4991-89be-b594d823843c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:45.168085 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 167/320] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:02:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:02:45.191002 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3434fa42-e131-4029-9543-21ddfa7cecda None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:45.296652 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3434fa42-e131-4029-9543-21ddfa7cecda None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:45.329447 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d12e7f-dd4c-4854-b5c1-717a22c46626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:45.330000 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d12e7f-dd4c-4854-b5c1-717a22c46626 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:45.330281 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d12e7f-dd4c-4854-b5c1-717a22c46626 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:45.330552 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d12e7f-dd4c-4854-b5c1-717a22c46626 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:45.340454 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d12e7f-dd4c-4854-b5c1-717a22c46626 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:02:45.345410 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-e4d12e7f-dd4c-4854-b5c1-717a22c46626 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 21:02:45.345410 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:02:45.345410 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:02:45.345410 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:02:45.345410 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:02:45.345410 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:02:45.345410 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:02:45.345410 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:02:45.345410 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:02:45.345410 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:02:45.345410 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:02:45.345410 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:02:45.345410 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:02:45.345410 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:02:45.345410 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:02:45.345410 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:02:45.346351 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:02:45.346351 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:02:45.346351 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:02:45.346351 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:02:45.346351 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:02:45.346351 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:02:45.346351 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:02:45.346351 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:02:45.346351 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:02:45.346351 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:02:45.346351 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:02:45.346351 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:02:45.346351 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:02:45.346351 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:02:45.346351 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:02:45.346351 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:02:45.348049 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:02:45.348049 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:02:45.348049 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:02:45.348049 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:02:45.348049 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:02:45.348049 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:02:45.348049 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:02:45.348049 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:02:45.348049 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:02:45.348049 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:02:45.348049 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 21:02:45.348049 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:02:45.351388 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 154/321] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Aug 4 21:02:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:02:45.353142 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44869f7-f21b-4991-89be-b594d823843c None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:45.371260 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:45.376600 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa5ce6b-bf61-4fd6-b1d2-27f3fde482a2 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:45.376887 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa5ce6b-bf61-4fd6-b1d2-27f3fde482a2 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:45.377160 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa5ce6b-bf61-4fd6-b1d2-27f3fde482a2 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:45.378555 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 155/322] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:02:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:02:45.383534 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feed8b71-fe42-4f9b-86c4-b5a2073c20ce None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:45.383882 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feed8b71-fe42-4f9b-86c4-b5a2073c20ce None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:45.384162 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feed8b71-fe42-4f9b-86c4-b5a2073c20ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:45.402716 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-feed8b71-fe42-4f9b-86c4-b5a2073c20ce None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:45.403951 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-feed8b71-fe42-4f9b-86c4-b5a2073c20ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:45.433601 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:45.458614 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44869f7-f21b-4991-89be-b594d823843c None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:45.477332 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 156/323] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:02:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 21:02:45.495837 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bae0f6-790a-44a1-90b2-5d66151bc3b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:45.496461 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bae0f6-790a-44a1-90b2-5d66151bc3b5 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:45.496744 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bae0f6-790a-44a1-90b2-5d66151bc3b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:45.497015 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bae0f6-790a-44a1-90b2-5d66151bc3b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:45.497789 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bae0f6-790a-44a1-90b2-5d66151bc3b5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:45.498089 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bae0f6-790a-44a1-90b2-5d66151bc3b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:45.499560 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bae0f6-790a-44a1-90b2-5d66151bc3b5 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:45.500168 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feed8b71-fe42-4f9b-86c4-b5a2073c20ce None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:45.513155 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 168/324] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Aug 4 21:02:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:45.514882 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bae0f6-790a-44a1-90b2-5d66151bc3b5 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:45.600514 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:45.605513 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feed8b71-fe42-4f9b-86c4-b5a2073c20ce None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:45.623707 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bae0f6-790a-44a1-90b2-5d66151bc3b5 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:45.646245 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d553346-532e-4468-92bf-182d63ca9900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:45.647267 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d553346-532e-4468-92bf-182d63ca9900 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:45.648110 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d553346-532e-4468-92bf-182d63ca9900 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:45.648807 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d553346-532e-4468-92bf-182d63ca9900 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:45.650271 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d553346-532e-4468-92bf-182d63ca9900 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:45.652434 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d553346-532e-4468-92bf-182d63ca9900 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:45.654918 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 157/325] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:02:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:02:45.659441 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86131c81-acd4-4cff-8773-10b26aeac95f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:45.659441 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86131c81-acd4-4cff-8773-10b26aeac95f None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:45.659628 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86131c81-acd4-4cff-8773-10b26aeac95f None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:45.659825 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86131c81-acd4-4cff-8773-10b26aeac95f None admin] PATH_INFO: `/v3/users/0e48df4352b9423c8195eeb07f854ff5` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:45.661074 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86131c81-acd4-4cff-8773-10b26aeac95f None admin] RBAC: Authorizing `identity:get_user(user_id=0e48df4352b9423c8195eeb07f854ff5)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:45.661176 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d553346-532e-4468-92bf-182d63ca9900 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:45.662512 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86131c81-acd4-4cff-8773-10b26aeac95f None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:45.664249 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 169/326] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Aug 4 21:02:45 2022] GET /identity/v3/users/0e48df4352b9423c8195eeb07f854ff5 => generated 293 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:02:45.670158 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86131c81-acd4-4cff-8773-10b26aeac95f None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:45.699660 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:45.773384 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d553346-532e-4468-92bf-182d63ca9900 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:45.776476 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86131c81-acd4-4cff-8773-10b26aeac95f None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:45.808852 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d99f82-dca3-4c77-9142-e7b1f6551e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:45.809475 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d99f82-dca3-4c77-9142-e7b1f6551e6a None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:45.809811 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d99f82-dca3-4c77-9142-e7b1f6551e6a None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:45.810117 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d99f82-dca3-4c77-9142-e7b1f6551e6a None admin] PATH_INFO: `/v3/groups` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:45.810859 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d99f82-dca3-4c77-9142-e7b1f6551e6a None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:45.812350 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d99f82-dca3-4c77-9142-e7b1f6551e6a None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:45.812643 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dff78f-c80d-459b-adb5-5ae01d04a727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:45.813268 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dff78f-c80d-459b-adb5-5ae01d04a727 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:45.813555 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dff78f-c80d-459b-adb5-5ae01d04a727 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:45.813833 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dff78f-c80d-459b-adb5-5ae01d04a727 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:45.814624 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dff78f-c80d-459b-adb5-5ae01d04a727 None admin] RBAC: Adding query filter params (role.id=8feebc60363c4feb870718b1721e5324, user.id=0e48df4352b9423c8195eeb07f854ff5, scope.system=all) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:45.814932 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dff78f-c80d-459b-adb5-5ae01d04a727 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:45.816935 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dff78f-c80d-459b-adb5-5ae01d04a727 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:45.829717 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 170/327] 10.222.0.25 () {56 vars in 1308 bytes} [Thu Aug 4 21:02:45 2022] GET /identity/v3/role_assignments?user.id=0e48df4352b9423c8195eeb07f854ff5&scope.system=all&role.id=8feebc60363c4feb870718b1721e5324 => generated 510 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:02:45.855400 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e7b06c-8571-4f59-b939-8f04216ba935 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:45.855706 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e7b06c-8571-4f59-b939-8f04216ba935 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:45.856051 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e7b06c-8571-4f59-b939-8f04216ba935 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:45.857634 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 171/328] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:02:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 21:02:45.863709 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44a7f89-0f7e-42ed-8270-65a95f05aabe None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:45.864065 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44a7f89-0f7e-42ed-8270-65a95f05aabe None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:45.864352 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44a7f89-0f7e-42ed-8270-65a95f05aabe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:45.871894 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:45.885654 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-d44a7f89-0f7e-42ed-8270-65a95f05aabe None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:45.886883 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-d44a7f89-0f7e-42ed-8270-65a95f05aabe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:45.923348 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 158/329] 10.222.0.25 () {60 vars in 1134 bytes} [Thu Aug 4 21:02:45 2022] POST /identity/v3/groups => generated 226 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:02:45.973429 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 172/330] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:02:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 21:02:45.996239 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d99f82-dca3-4c77-9142-e7b1f6551e6a None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:46.098362 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:46.112720 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d99f82-dca3-4c77-9142-e7b1f6551e6a None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:46.140791 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c94944c-e425-448c-b32a-add998a6af31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:46.143059 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c94944c-e425-448c-b32a-add998a6af31 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:46.143059 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c94944c-e425-448c-b32a-add998a6af31 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:46.143059 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c94944c-e425-448c-b32a-add998a6af31 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:46.153035 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c94944c-e425-448c-b32a-add998a6af31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:02:46.158018 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-9c94944c-e425-448c-b32a-add998a6af31 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 21:02:46.158018 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:02:46.158018 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:02:46.158018 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:02:46.158018 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:02:46.158018 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:02:46.158018 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:02:46.158018 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:02:46.158018 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:02:46.158018 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:02:46.158018 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:02:46.158018 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:02:46.158018 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:02:46.158018 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:02:46.158018 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:02:46.158018 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:02:46.159450 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:02:46.159450 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:02:46.159450 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:02:46.159450 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:02:46.159450 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:02:46.159450 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:02:46.159450 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:02:46.159450 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:02:46.159450 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:02:46.159450 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:02:46.159450 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:02:46.159450 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:02:46.159450 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:02:46.159450 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:02:46.159450 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:02:46.159450 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:02:46.160369 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:02:46.160369 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:02:46.160369 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:02:46.160369 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:02:46.160369 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:02:46.160369 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:02:46.160369 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:02:46.160369 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:02:46.160369 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:02:46.160369 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:02:46.160369 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 21:02:46.160369 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:02:46.160369 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 159/331] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Aug 4 21:02:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:02:46.165550 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44a7f89-0f7e-42ed-8270-65a95f05aabe None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:46.278893 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44a7f89-0f7e-42ed-8270-65a95f05aabe None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:46.321778 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10d07a2-52d5-49fd-9b47-9efab235c3af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:46.322408 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10d07a2-52d5-49fd-9b47-9efab235c3af None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:46.322693 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10d07a2-52d5-49fd-9b47-9efab235c3af None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:46.322968 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10d07a2-52d5-49fd-9b47-9efab235c3af None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:46.323767 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10d07a2-52d5-49fd-9b47-9efab235c3af None admin] RBAC: Adding query filter params (name=reader) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:46.324072 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10d07a2-52d5-49fd-9b47-9efab235c3af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:46.325445 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10d07a2-52d5-49fd-9b47-9efab235c3af None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:46.326989 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:46.338532 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 173/332] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Aug 4 21:02:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:46.344336 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c94944c-e425-448c-b32a-add998a6af31 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:46.461960 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c94944c-e425-448c-b32a-add998a6af31 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:46.502021 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d29677-1374-4617-b367-12c7619b678d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:46.502672 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d29677-1374-4617-b367-12c7619b678d None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:46.503555 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d29677-1374-4617-b367-12c7619b678d None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:46.503831 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d29677-1374-4617-b367-12c7619b678d None admin] PATH_INFO: `/v3/users/bb0b5e8554204087a31b901e2195eb1f` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:46.505012 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d29677-1374-4617-b367-12c7619b678d None admin] RBAC: Authorizing `identity:get_user(user_id=bb0b5e8554204087a31b901e2195eb1f)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:46.506487 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d29677-1374-4617-b367-12c7619b678d None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:46.508295 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 160/333] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Aug 4 21:02:46 2022] GET /identity/v3/users/bb0b5e8554204087a31b901e2195eb1f => generated 293 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:46.513982 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10d07a2-52d5-49fd-9b47-9efab235c3af None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:46.632521 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10d07a2-52d5-49fd-9b47-9efab235c3af None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:46.669087 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38bfca8-4477-401a-b91c-b84eae572780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:46.670177 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38bfca8-4477-401a-b91c-b84eae572780 None admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:46.670683 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38bfca8-4477-401a-b91c-b84eae572780 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:46.671158 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38bfca8-4477-401a-b91c-b84eae572780 None admin] PATH_INFO: `/v3/system/users/bb0b5e8554204087a31b901e2195eb1f/roles/5ab0887d24bc41b882f6786b7aab0fe8` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:46.672884 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38bfca8-4477-401a-b91c-b84eae572780 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=bb0b5e8554204087a31b901e2195eb1f, role_id=5ab0887d24bc41b882f6786b7aab0fe8)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:46.674470 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38bfca8-4477-401a-b91c-b84eae572780 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:46.742538 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 174/334] 10.222.0.25 () {58 vars in 1334 bytes} [Thu Aug 4 21:02:46 2022] PUT /identity/v3/system/users/bb0b5e8554204087a31b901e2195eb1f/roles/5ab0887d24bc41b882f6786b7aab0fe8 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:02:48.248531 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b484851-db36-4d05-835c-8162bf80cf83 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:48.249026 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b484851-db36-4d05-835c-8162bf80cf83 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:48.249129 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b484851-db36-4d05-835c-8162bf80cf83 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:48.250646 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 161/335] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:02:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:02:48.256686 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711e344c-e169-437f-926d-5ac29e962255 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:48.257505 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711e344c-e169-437f-926d-5ac29e962255 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:48.257983 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711e344c-e169-437f-926d-5ac29e962255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:48.280699 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-711e344c-e169-437f-926d-5ac29e962255 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:48.282279 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-711e344c-e169-437f-926d-5ac29e962255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:48.350984 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:48.364298 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 175/336] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:02:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:02:48.387839 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b484851-db36-4d05-835c-8162bf80cf83 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:48.416781 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:48.513034 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b484851-db36-4d05-835c-8162bf80cf83 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:48.556981 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630d7c27-c02b-43c2-8924-66af05de06e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:48.557625 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630d7c27-c02b-43c2-8924-66af05de06e2 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:48.557904 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630d7c27-c02b-43c2-8924-66af05de06e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:48.558172 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630d7c27-c02b-43c2-8924-66af05de06e2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:48.572074 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630d7c27-c02b-43c2-8924-66af05de06e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:02:48.576994 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-630d7c27-c02b-43c2-8924-66af05de06e2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 21:02:48.576994 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:02:48.576994 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:02:48.576994 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:02:48.576994 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:02:48.576994 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:02:48.576994 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:02:48.576994 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:02:48.576994 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:02:48.576994 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:02:48.576994 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:02:48.576994 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:02:48.576994 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:02:48.576994 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:02:48.576994 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:02:48.576994 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:02:48.577866 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:02:48.577866 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:02:48.577866 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:02:48.577866 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:02:48.577866 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:02:48.577866 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:02:48.577866 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:02:48.577866 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:02:48.577866 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:02:48.577866 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:02:48.577866 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:02:48.577866 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:02:48.577866 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:02:48.577866 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:02:48.577866 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:02:48.577866 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:02:48.578663 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:02:48.578663 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:02:48.578663 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:02:48.578663 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:02:48.578663 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:02:48.578663 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:02:48.578663 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:02:48.578663 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:02:48.578663 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:02:48.578663 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:02:48.578663 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 21:02:48.578663 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:02:48.578663 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 162/337] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Aug 4 21:02:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:02:48.584549 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711e344c-e169-437f-926d-5ac29e962255 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:48.717967 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711e344c-e169-437f-926d-5ac29e962255 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:48.771759 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c98d88-a454-44c5-8d79-90e9264af5d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:48.772738 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c98d88-a454-44c5-8d79-90e9264af5d4 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:48.774122 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c98d88-a454-44c5-8d79-90e9264af5d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:48.774603 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c98d88-a454-44c5-8d79-90e9264af5d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:48.775584 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c98d88-a454-44c5-8d79-90e9264af5d4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:48.776093 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c98d88-a454-44c5-8d79-90e9264af5d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:48.777678 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c98d88-a454-44c5-8d79-90e9264af5d4 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:48.791700 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 176/338] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Aug 4 21:02:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:48.797735 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630d7c27-c02b-43c2-8924-66af05de06e2 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:48.915697 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:48.929657 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630d7c27-c02b-43c2-8924-66af05de06e2 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:48.974883 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75566e26-3aa8-491b-ba8d-c5bbfdb2a042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:48.975541 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75566e26-3aa8-491b-ba8d-c5bbfdb2a042 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:48.975822 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75566e26-3aa8-491b-ba8d-c5bbfdb2a042 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:48.976094 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75566e26-3aa8-491b-ba8d-c5bbfdb2a042 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:49.069543 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75566e26-3aa8-491b-ba8d-c5bbfdb2a042 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:49.070966 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75566e26-3aa8-491b-ba8d-c5bbfdb2a042 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:49.171919 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:49.194212 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 163/339] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Aug 4 21:02:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 21:02:49.200416 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c98d88-a454-44c5-8d79-90e9264af5d4 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:49.292841 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:49.359699 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c98d88-a454-44c5-8d79-90e9264af5d4 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:49.388654 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2215e192-d553-4e2c-931f-e59919111c38 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:49.389155 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2215e192-d553-4e2c-931f-e59919111c38 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:49.389229 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2215e192-d553-4e2c-931f-e59919111c38 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:49.390547 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 164/340] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:02:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:02:49.395735 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041a6935-71b3-49ab-8500-e98fea260d29 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:49.396016 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041a6935-71b3-49ab-8500-e98fea260d29 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:49.396292 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041a6935-71b3-49ab-8500-e98fea260d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:49.403853 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6fcd63-58fc-46d4-bc52-a5e5dc9460a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:49.404804 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6fcd63-58fc-46d4-bc52-a5e5dc9460a8 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:49.405287 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6fcd63-58fc-46d4-bc52-a5e5dc9460a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:49.405873 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6fcd63-58fc-46d4-bc52-a5e5dc9460a8 None admin] PATH_INFO: `/v3/projects/b8f804a236884f039371233cce01ecef` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:49.407413 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6fcd63-58fc-46d4-bc52-a5e5dc9460a8 None admin] RBAC: Authorizing `identity:get_project(project_id=b8f804a236884f039371233cce01ecef)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:49.409361 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6fcd63-58fc-46d4-bc52-a5e5dc9460a8 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:49.411654 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 177/341] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Aug 4 21:02:49 2022] GET /identity/v3/projects/b8f804a236884f039371233cce01ecef => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:49.413087 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653e4798-96a2-4d5b-9db5-2fd9da69b4a3 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:49.413660 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653e4798-96a2-4d5b-9db5-2fd9da69b4a3 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:49.414128 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653e4798-96a2-4d5b-9db5-2fd9da69b4a3 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:49.415885 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 178/342] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:02:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:02:49.416787 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653e4798-96a2-4d5b-9db5-2fd9da69b4a3 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:49.434544 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-041a6935-71b3-49ab-8500-e98fea260d29 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:49.436142 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-041a6935-71b3-49ab-8500-e98fea260d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:49.520268 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 165/343] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:02:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 21:02:49.522082 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c4f7d-1889-4c09-8ff1-d3d5b70277d3 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:49.522369 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c4f7d-1889-4c09-8ff1-d3d5b70277d3 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:49.522646 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c4f7d-1889-4c09-8ff1-d3d5b70277d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:49.541862 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-2d6c4f7d-1889-4c09-8ff1-d3d5b70277d3 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:49.543228 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-2d6c4f7d-1889-4c09-8ff1-d3d5b70277d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:49.609884 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653e4798-96a2-4d5b-9db5-2fd9da69b4a3 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:49.616134 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 166/344] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:02:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:02:49.617379 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9983f3a2-1059-4d69-911b-1991131c29d9 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:49.617673 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9983f3a2-1059-4d69-911b-1991131c29d9 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:49.617953 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9983f3a2-1059-4d69-911b-1991131c29d9 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:49.619224 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 167/345] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:02:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:02:49.624587 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9983f3a2-1059-4d69-911b-1991131c29d9 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:49.646768 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:49.670013 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:49.671979 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489afdbd-2e52-408b-a752-eed7a7b223ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:49.672698 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489afdbd-2e52-408b-a752-eed7a7b223ac None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:49.673080 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489afdbd-2e52-408b-a752-eed7a7b223ac None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:49.673354 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489afdbd-2e52-408b-a752-eed7a7b223ac None admin] PATH_INFO: `/v3/groups/ba5e1c67c05e4d9582187eb89cd1acb5` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:49.683673 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489afdbd-2e52-408b-a752-eed7a7b223ac None admin] RBAC: Authorizing `identity:get_group(group_id=ba5e1c67c05e4d9582187eb89cd1acb5)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:49.685143 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489afdbd-2e52-408b-a752-eed7a7b223ac None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:49.687151 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 179/346] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Aug 4 21:02:49 2022] GET /identity/v3/groups/ba5e1c67c05e4d9582187eb89cd1acb5 => generated 226 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:49.688534 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48e884e-0c3f-438a-9efc-ad0e56a9716b None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:49.688885 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48e884e-0c3f-438a-9efc-ad0e56a9716b None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:49.689229 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48e884e-0c3f-438a-9efc-ad0e56a9716b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:49.709686 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-e48e884e-0c3f-438a-9efc-ad0e56a9716b None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:49.711058 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-e48e884e-0c3f-438a-9efc-ad0e56a9716b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:49.733874 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9983f3a2-1059-4d69-911b-1991131c29d9 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:49.771641 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c61e0df-6b75-4baa-b05c-db2cfafa6b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:49.772302 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c61e0df-6b75-4baa-b05c-db2cfafa6b3d None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:49.772591 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c61e0df-6b75-4baa-b05c-db2cfafa6b3d None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:49.772875 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c61e0df-6b75-4baa-b05c-db2cfafa6b3d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:49.784435 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c61e0df-6b75-4baa-b05c-db2cfafa6b3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:02:49.785276 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 180/347] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:02:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:02:49.786111 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48e884e-0c3f-438a-9efc-ad0e56a9716b None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:49.796810 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-1c61e0df-6b75-4baa-b05c-db2cfafa6b3d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 21:02:49.796810 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:02:49.796810 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:02:49.796810 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:02:49.796810 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:02:49.796810 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:02:49.796810 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:02:49.796810 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:02:49.796810 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:02:49.796810 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:02:49.796810 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:02:49.796810 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:02:49.796810 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:02:49.796810 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:02:49.796810 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:02:49.796810 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:02:49.797744 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:02:49.797744 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:02:49.797744 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:02:49.797744 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:02:49.797744 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:02:49.797744 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:02:49.797744 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:02:49.797744 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:02:49.797744 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:02:49.797744 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:02:49.797744 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:02:49.797744 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:02:49.797744 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:02:49.797744 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:02:49.797744 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:02:49.797744 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:02:49.798530 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:02:49.798530 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:02:49.798530 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:02:49.798530 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:02:49.798530 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:02:49.798530 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:02:49.798530 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:02:49.798530 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:02:49.798530 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:02:49.798530 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:02:49.798530 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 21:02:49.798530 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:02:49.798530 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 168/348] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Aug 4 21:02:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:02:49.799186 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c61e0df-6b75-4baa-b05c-db2cfafa6b3d None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:49.892021 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48e884e-0c3f-438a-9efc-ad0e56a9716b None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:49.908182 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:49.918794 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c61e0df-6b75-4baa-b05c-db2cfafa6b3d None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:49.930066 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:49.931118 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096fae38-393b-4c3a-a6d5-ee3f606473aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:49.931824 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096fae38-393b-4c3a-a6d5-ee3f606473aa None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:49.932093 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096fae38-393b-4c3a-a6d5-ee3f606473aa None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:49.932365 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096fae38-393b-4c3a-a6d5-ee3f606473aa None admin] PATH_INFO: `/v3/roles/member` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:49.945928 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096fae38-393b-4c3a-a6d5-ee3f606473aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:02:49.951284 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-096fae38-393b-4c3a-a6d5-ee3f606473aa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 21:02:49.951284 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:02:49.951284 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:02:49.951284 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:02:49.951284 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:02:49.951284 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:02:49.951284 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:02:49.951284 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:02:49.951284 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:02:49.951284 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:02:49.951284 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:02:49.951284 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:02:49.951284 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:02:49.951284 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:02:49.951284 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:02:49.951284 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:02:49.952307 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:02:49.952307 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:02:49.952307 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:02:49.952307 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:02:49.952307 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:02:49.952307 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:02:49.952307 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:02:49.952307 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:02:49.952307 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:02:49.952307 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:02:49.952307 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:02:49.952307 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:02:49.952307 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:02:49.952307 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:02:49.952307 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:02:49.952307 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:02:49.953120 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:02:49.953120 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:02:49.953120 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:02:49.953120 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:02:49.953120 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:02:49.953120 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:02:49.953120 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:02:49.953120 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:02:49.953120 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:02:49.953120 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:02:49.953120 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 21:02:49.953120 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:02:49.953120 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 181/349] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Aug 4 21:02:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:02:49.953795 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096fae38-393b-4c3a-a6d5-ee3f606473aa None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:49.956858 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d07ba2-4033-42f8-b32b-e7579749a3e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:49.957483 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d07ba2-4033-42f8-b32b-e7579749a3e8 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:49.957830 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d07ba2-4033-42f8-b32b-e7579749a3e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:49.958109 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d07ba2-4033-42f8-b32b-e7579749a3e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:49.958956 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d07ba2-4033-42f8-b32b-e7579749a3e8 None admin] RBAC: Adding query filter params (group.id=ba5e1c67c05e4d9582187eb89cd1acb5, role.id=8feebc60363c4feb870718b1721e5324, scope.project.id=b8f804a236884f039371233cce01ecef) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:49.959273 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d07ba2-4033-42f8-b32b-e7579749a3e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:49.960994 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d07ba2-4033-42f8-b32b-e7579749a3e8 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:49.967578 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 169/350] 10.222.0.25 () {56 vars in 1376 bytes} [Thu Aug 4 21:02:49 2022] GET /identity/v3/role_assignments?group.id=ba5e1c67c05e4d9582187eb89cd1acb5&scope.project.id=b8f804a236884f039371233cce01ecef&role.id=8feebc60363c4feb870718b1721e5324 => generated 260 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:49.968735 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d07ba2-4033-42f8-b32b-e7579749a3e8 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:50.074062 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096fae38-393b-4c3a-a6d5-ee3f606473aa None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:50.085694 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d07ba2-4033-42f8-b32b-e7579749a3e8 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:50.117062 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ba3acd-d86f-4ca1-8b60-879db641fbb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:50.117692 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ba3acd-d86f-4ca1-8b60-879db641fbb6 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:50.117970 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ba3acd-d86f-4ca1-8b60-879db641fbb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:50.118236 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ba3acd-d86f-4ca1-8b60-879db641fbb6 None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:50.118958 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ba3acd-d86f-4ca1-8b60-879db641fbb6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:50.119259 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ba3acd-d86f-4ca1-8b60-879db641fbb6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:50.120852 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ba3acd-d86f-4ca1-8b60-879db641fbb6 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:50.128491 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d73102-cba5-4829-86c1-d5b4a66bcbce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:50.129531 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d73102-cba5-4829-86c1-d5b4a66bcbce None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:50.129887 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d73102-cba5-4829-86c1-d5b4a66bcbce None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:50.130232 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d73102-cba5-4829-86c1-d5b4a66bcbce None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:50.130914 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 182/351] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Aug 4 21:02:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:50.131810 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ba3acd-d86f-4ca1-8b60-879db641fbb6 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:50.143938 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d73102-cba5-4829-86c1-d5b4a66bcbce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:02:50.149912 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-80d73102-cba5-4829-86c1-d5b4a66bcbce None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 21:02:50.149912 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:02:50.149912 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:02:50.149912 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:02:50.149912 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:02:50.149912 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:02:50.149912 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:02:50.149912 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:02:50.149912 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:02:50.149912 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:02:50.149912 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:02:50.149912 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:02:50.149912 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:02:50.149912 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:02:50.149912 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:02:50.149912 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:02:50.150822 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:02:50.150822 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:02:50.150822 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:02:50.150822 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:02:50.150822 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:02:50.150822 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:02:50.150822 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:02:50.150822 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:02:50.150822 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:02:50.150822 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:02:50.150822 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:02:50.150822 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:02:50.150822 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:02:50.150822 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:02:50.150822 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:02:50.150822 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:02:50.151673 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:02:50.151673 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:02:50.151673 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:02:50.151673 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:02:50.151673 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:02:50.151673 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:02:50.151673 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:02:50.151673 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:02:50.151673 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:02:50.151673 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:02:50.151673 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 21:02:50.151673 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:02:50.151673 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 170/352] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Aug 4 21:02:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:02:50.152341 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d73102-cba5-4829-86c1-d5b4a66bcbce None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:50.171309 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:50.215726 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:50.248231 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ba3acd-d86f-4ca1-8b60-879db641fbb6 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:50.273318 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d73102-cba5-4829-86c1-d5b4a66bcbce None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:50.289930 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4966ec7b-e45d-4059-bb37-a348c4b90f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:50.290647 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4966ec7b-e45d-4059-bb37-a348c4b90f5c None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:50.290928 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4966ec7b-e45d-4059-bb37-a348c4b90f5c None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:50.291244 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4966ec7b-e45d-4059-bb37-a348c4b90f5c None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:50.293151 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4966ec7b-e45d-4059-bb37-a348c4b90f5c None admin] RBAC: Adding query filter params (name=member) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:50.293151 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4966ec7b-e45d-4059-bb37-a348c4b90f5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:50.293748 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4966ec7b-e45d-4059-bb37-a348c4b90f5c None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:50.305992 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 183/353] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Aug 4 21:02:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:50.306887 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4966ec7b-e45d-4059-bb37-a348c4b90f5c None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:50.318163 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec27563-6449-4350-8439-39d47d5021cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:50.318800 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec27563-6449-4350-8439-39d47d5021cd None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:50.319089 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec27563-6449-4350-8439-39d47d5021cd None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:50.319394 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec27563-6449-4350-8439-39d47d5021cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:50.398395 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec27563-6449-4350-8439-39d47d5021cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:50.399955 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec27563-6449-4350-8439-39d47d5021cd None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:50.432205 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:50.436464 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4966ec7b-e45d-4059-bb37-a348c4b90f5c None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:50.439082 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:50.480467 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6395a0ff-9ccc-4fb1-8b9a-d0254a6fdea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:50.481102 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6395a0ff-9ccc-4fb1-8b9a-d0254a6fdea8 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:50.481385 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6395a0ff-9ccc-4fb1-8b9a-d0254a6fdea8 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:50.481675 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6395a0ff-9ccc-4fb1-8b9a-d0254a6fdea8 None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:50.482386 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6395a0ff-9ccc-4fb1-8b9a-d0254a6fdea8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:50.482690 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6395a0ff-9ccc-4fb1-8b9a-d0254a6fdea8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:50.484129 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6395a0ff-9ccc-4fb1-8b9a-d0254a6fdea8 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:50.492545 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 184/354] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Aug 4 21:02:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:50.495017 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1829509-e747-459b-8a3c-a08120582d35 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:50.495309 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1829509-e747-459b-8a3c-a08120582d35 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:50.495609 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1829509-e747-459b-8a3c-a08120582d35 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:50.500713 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 185/355] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:02:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:02:50.500713 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1829509-e747-459b-8a3c-a08120582d35 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:50.544217 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 171/356] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Aug 4 21:02:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 21:02:50.546852 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198f2cf9-d1cf-4436-872f-c47362bcc914 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:50.547407 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198f2cf9-d1cf-4436-872f-c47362bcc914 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:50.547878 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198f2cf9-d1cf-4436-872f-c47362bcc914 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:50.549333 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 172/357] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:02:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:02:50.550254 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198f2cf9-d1cf-4436-872f-c47362bcc914 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:50.639744 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1829509-e747-459b-8a3c-a08120582d35 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:50.679154 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fa8c6a-79b3-4beb-aaa1-872c0d9ebc4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:50.680130 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fa8c6a-79b3-4beb-aaa1-872c0d9ebc4a None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:50.680585 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fa8c6a-79b3-4beb-aaa1-872c0d9ebc4a None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:50.681322 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fa8c6a-79b3-4beb-aaa1-872c0d9ebc4a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:50.682170 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198f2cf9-d1cf-4436-872f-c47362bcc914 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:50.710043 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:50.716057 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:50.722454 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c62b5b8-fd05-4525-9e10-abed86a168d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:50.723352 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c62b5b8-fd05-4525-9e10-abed86a168d0 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:50.723795 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c62b5b8-fd05-4525-9e10-abed86a168d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:50.728222 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c62b5b8-fd05-4525-9e10-abed86a168d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:50.774650 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fa8c6a-79b3-4beb-aaa1-872c0d9ebc4a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:50.776650 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fa8c6a-79b3-4beb-aaa1-872c0d9ebc4a None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:50.826368 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c62b5b8-fd05-4525-9e10-abed86a168d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:50.828961 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c62b5b8-fd05-4525-9e10-abed86a168d0 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:50.902353 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 186/358] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Aug 4 21:02:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 21:02:50.904070 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c842f11-c7ed-4d5e-bda1-f948486828e5 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:50.904467 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c842f11-c7ed-4d5e-bda1-f948486828e5 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:50.904813 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c842f11-c7ed-4d5e-bda1-f948486828e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:50.925559 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-7c842f11-c7ed-4d5e-bda1-f948486828e5 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:50.926892 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-7c842f11-c7ed-4d5e-bda1-f948486828e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:50.966882 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 173/359] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Aug 4 21:02:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 21:02:50.967693 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c62b5b8-fd05-4525-9e10-abed86a168d0 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:50.998172 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:51.004545 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:51.019563 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 187/360] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:02:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:02:51.020915 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6111f1a5-9870-483a-aa07-c18706c6d4ba None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:51.026127 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6111f1a5-9870-483a-aa07-c18706c6d4ba None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:51.026127 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6111f1a5-9870-483a-aa07-c18706c6d4ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:51.040417 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-6111f1a5-9870-483a-aa07-c18706c6d4ba None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:51.041613 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-6111f1a5-9870-483a-aa07-c18706c6d4ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:51.093944 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c62b5b8-fd05-4525-9e10-abed86a168d0 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:51.129210 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 188/361] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:02:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:02:51.130162 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6111f1a5-9870-483a-aa07-c18706c6d4ba None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:51.142021 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e30fa44-6146-4d69-8299-bd27179c561e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:51.143025 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e30fa44-6146-4d69-8299-bd27179c561e None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:51.143671 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e30fa44-6146-4d69-8299-bd27179c561e None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:51.144175 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e30fa44-6146-4d69-8299-bd27179c561e None admin] PATH_INFO: `/v3/projects/b8f804a236884f039371233cce01ecef` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:51.146653 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e30fa44-6146-4d69-8299-bd27179c561e None admin] RBAC: Authorizing `identity:get_project(project_id=b8f804a236884f039371233cce01ecef)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:51.153942 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e30fa44-6146-4d69-8299-bd27179c561e None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:51.153942 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 174/362] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Aug 4 21:02:50 2022] GET /identity/v3/projects/b8f804a236884f039371233cce01ecef => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:51.153942 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e30fa44-6146-4d69-8299-bd27179c561e None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:51.280949 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6111f1a5-9870-483a-aa07-c18706c6d4ba None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:51.291223 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:51.303757 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e30fa44-6146-4d69-8299-bd27179c561e None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:51.307281 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:51.331540 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bbd65d-08e2-4ead-bfa6-52bea01263b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:51.332261 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bbd65d-08e2-4ead-bfa6-52bea01263b8 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:51.332557 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bbd65d-08e2-4ead-bfa6-52bea01263b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:51.332831 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bbd65d-08e2-4ead-bfa6-52bea01263b8 None admin] PATH_INFO: `/v3/projects/6c2de9abe1504f6698a9ea2a032c91e4` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:51.334225 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bbd65d-08e2-4ead-bfa6-52bea01263b8 None admin] RBAC: Authorizing `identity:get_project(project_id=6c2de9abe1504f6698a9ea2a032c91e4)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:51.336105 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bbd65d-08e2-4ead-bfa6-52bea01263b8 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:51.338149 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 189/363] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Aug 4 21:02:51 2022] GET /identity/v3/projects/6c2de9abe1504f6698a9ea2a032c91e4 => generated 302 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:51.340097 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bbd65d-08e2-4ead-bfa6-52bea01263b8 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:51.349410 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29527a77-a1d7-4b60-bccd-29c505a29957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:51.350083 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29527a77-a1d7-4b60-bccd-29c505a29957 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:51.350369 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29527a77-a1d7-4b60-bccd-29c505a29957 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:51.350643 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29527a77-a1d7-4b60-bccd-29c505a29957 None admin] PATH_INFO: `/v3/projects/6c2de9abe1504f6698a9ea2a032c91e4` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:51.351920 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29527a77-a1d7-4b60-bccd-29c505a29957 None admin] RBAC: Authorizing `identity:get_project(project_id=6c2de9abe1504f6698a9ea2a032c91e4)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:51.353417 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29527a77-a1d7-4b60-bccd-29c505a29957 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:51.355172 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 175/364] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Aug 4 21:02:51 2022] GET /identity/v3/projects/6c2de9abe1504f6698a9ea2a032c91e4 => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:51.355984 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29527a77-a1d7-4b60-bccd-29c505a29957 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:51.485569 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29527a77-a1d7-4b60-bccd-29c505a29957 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:51.492757 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bbd65d-08e2-4ead-bfa6-52bea01263b8 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:51.530698 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c408a6cb-71d5-4fec-9cbb-88f8ff800789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:51.531717 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c408a6cb-71d5-4fec-9cbb-88f8ff800789 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:51.532246 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c408a6cb-71d5-4fec-9cbb-88f8ff800789 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:51.532802 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c408a6cb-71d5-4fec-9cbb-88f8ff800789 None admin] PATH_INFO: `/v3/groups/ba5e1c67c05e4d9582187eb89cd1acb5` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:51.534501 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c408a6cb-71d5-4fec-9cbb-88f8ff800789 None admin] RBAC: Authorizing `identity:get_group(group_id=ba5e1c67c05e4d9582187eb89cd1acb5)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:51.536203 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c408a6cb-71d5-4fec-9cbb-88f8ff800789 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:51.537123 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f1fa53-6cf0-4648-9b29-aedc4ffe6326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:51.537770 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f1fa53-6cf0-4648-9b29-aedc4ffe6326 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:51.538245 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 176/365] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Aug 4 21:02:51 2022] GET /identity/v3/groups/ba5e1c67c05e4d9582187eb89cd1acb5 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:51.538350 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f1fa53-6cf0-4648-9b29-aedc4ffe6326 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:51.538350 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f1fa53-6cf0-4648-9b29-aedc4ffe6326 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:51.538803 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c408a6cb-71d5-4fec-9cbb-88f8ff800789 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:51.549390 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f1fa53-6cf0-4648-9b29-aedc4ffe6326 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:02:51.555522 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-d1f1fa53-6cf0-4648-9b29-aedc4ffe6326 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 21:02:51.555522 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:02:51.555522 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:02:51.555522 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:02:51.555522 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:02:51.555522 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:02:51.555522 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:02:51.555522 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:02:51.555522 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:02:51.555522 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:02:51.555522 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:02:51.555522 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:02:51.555522 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:02:51.555522 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:02:51.555522 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:02:51.555522 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:02:51.556455 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:02:51.556455 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:02:51.556455 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:02:51.556455 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:02:51.556455 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:02:51.556455 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:02:51.556455 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:02:51.556455 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:02:51.556455 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:02:51.556455 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:02:51.556455 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:02:51.556455 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:02:51.556455 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:02:51.556455 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:02:51.556455 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:02:51.556455 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:02:51.557648 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:02:51.557648 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:02:51.557648 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:02:51.557648 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:02:51.557648 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:02:51.557648 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:02:51.557648 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:02:51.557648 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:02:51.557648 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:02:51.557648 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:02:51.557648 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 21:02:51.557648 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:02:51.557648 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 190/366] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Aug 4 21:02:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:02:51.557648 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f1fa53-6cf0-4648-9b29-aedc4ffe6326 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:51.579287 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:51.632434 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:51.672756 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c408a6cb-71d5-4fec-9cbb-88f8ff800789 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:51.689745 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f1fa53-6cf0-4648-9b29-aedc4ffe6326 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:51.729301 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7956a2e9-c702-49ef-9301-6657b5f957cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:51.729955 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7956a2e9-c702-49ef-9301-6657b5f957cf None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:51.730239 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7956a2e9-c702-49ef-9301-6657b5f957cf None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:51.730513 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7956a2e9-c702-49ef-9301-6657b5f957cf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:51.736233 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e09f6c-4611-43f9-9b4b-a3f961c8d7fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:51.736233 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e09f6c-4611-43f9-9b4b-a3f961c8d7fc None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:51.736233 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e09f6c-4611-43f9-9b4b-a3f961c8d7fc None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:51.736233 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e09f6c-4611-43f9-9b4b-a3f961c8d7fc None admin] PATH_INFO: `/v3/groups/ba5e1c67c05e4d9582187eb89cd1acb5` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:51.736794 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e09f6c-4611-43f9-9b4b-a3f961c8d7fc None admin] RBAC: Authorizing `identity:get_group(group_id=ba5e1c67c05e4d9582187eb89cd1acb5)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:51.738338 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e09f6c-4611-43f9-9b4b-a3f961c8d7fc None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:51.740739 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 191/367] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Aug 4 21:02:51 2022] GET /identity/v3/groups/ba5e1c67c05e4d9582187eb89cd1acb5 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:51.741587 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e09f6c-4611-43f9-9b4b-a3f961c8d7fc None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:51.743951 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7956a2e9-c702-49ef-9301-6657b5f957cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:02:51.749028 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-7956a2e9-c702-49ef-9301-6657b5f957cf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:02:51.749028 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:02:51.749028 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:02:51.749028 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:02:51.749028 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:02:51.749028 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:02:51.749028 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:02:51.749028 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:02:51.749028 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:02:51.749028 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:02:51.749028 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:02:51.749028 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:02:51.749028 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:02:51.749028 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:02:51.749028 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:02:51.749028 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:02:51.749924 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:02:51.749924 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:02:51.749924 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:02:51.749924 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:02:51.749924 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:02:51.749924 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:02:51.749924 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:02:51.749924 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:02:51.749924 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:02:51.749924 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:02:51.749924 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:02:51.749924 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:02:51.749924 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:02:51.749924 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:02:51.749924 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:02:51.749924 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:02:51.750693 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:02:51.750693 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:02:51.750693 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:02:51.750693 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:02:51.750693 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:02:51.750693 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:02:51.750693 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:02:51.750693 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:02:51.750693 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:02:51.750693 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:02:51.750693 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:02:51.750693 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:02:51.750693 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 177/368] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Aug 4 21:02:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:02:51.751371 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7956a2e9-c702-49ef-9301-6657b5f957cf None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:51.857873 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:51.863241 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e09f6c-4611-43f9-9b4b-a3f961c8d7fc None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:51.869736 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7956a2e9-c702-49ef-9301-6657b5f957cf None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:51.892441 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:51.912031 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e60716-288e-4a2a-aef8-a3a945299ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:51.912623 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e60716-288e-4a2a-aef8-a3a945299ee1 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:51.912906 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e60716-288e-4a2a-aef8-a3a945299ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:51.913175 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e60716-288e-4a2a-aef8-a3a945299ee1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:51.913998 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e60716-288e-4a2a-aef8-a3a945299ee1 None admin] RBAC: Adding query filter params (group.id=ba5e1c67c05e4d9582187eb89cd1acb5, role.id=70587326d8584316b73b380df2ae8e67, scope.project.id=b8f804a236884f039371233cce01ecef) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:51.914308 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e60716-288e-4a2a-aef8-a3a945299ee1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:51.916284 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2abc6aa-74f6-4725-8030-42d01f5474fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:51.916404 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e60716-288e-4a2a-aef8-a3a945299ee1 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:51.916491 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2abc6aa-74f6-4725-8030-42d01f5474fd None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:51.916491 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2abc6aa-74f6-4725-8030-42d01f5474fd None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:51.916624 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2abc6aa-74f6-4725-8030-42d01f5474fd None admin] PATH_INFO: `/v3/groups/ba5e1c67c05e4d9582187eb89cd1acb5` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:51.917901 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2abc6aa-74f6-4725-8030-42d01f5474fd None admin] RBAC: Authorizing `identity:get_group(group_id=ba5e1c67c05e4d9582187eb89cd1acb5)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:51.920231 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2abc6aa-74f6-4725-8030-42d01f5474fd None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:51.921738 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 178/369] 10.222.0.25 () {56 vars in 1376 bytes} [Thu Aug 4 21:02:51 2022] GET /identity/v3/role_assignments?group.id=ba5e1c67c05e4d9582187eb89cd1acb5&scope.project.id=b8f804a236884f039371233cce01ecef&role.id=70587326d8584316b73b380df2ae8e67 => generated 260 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:51.925087 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 192/370] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Aug 4 21:02:51 2022] GET /identity/v3/groups/ba5e1c67c05e4d9582187eb89cd1acb5 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:51.925234 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e60716-288e-4a2a-aef8-a3a945299ee1 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:51.925333 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2abc6aa-74f6-4725-8030-42d01f5474fd None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:52.074582 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e60716-288e-4a2a-aef8-a3a945299ee1 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:52.082621 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2abc6aa-74f6-4725-8030-42d01f5474fd None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:52.124519 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f989ab62-5b8d-4aef-97b5-1be000f7afc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:52.125348 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f989ab62-5b8d-4aef-97b5-1be000f7afc9 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:52.125584 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f989ab62-5b8d-4aef-97b5-1be000f7afc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:52.125861 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f989ab62-5b8d-4aef-97b5-1be000f7afc9 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:52.126590 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f989ab62-5b8d-4aef-97b5-1be000f7afc9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:52.126898 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f989ab62-5b8d-4aef-97b5-1be000f7afc9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:52.128435 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f989ab62-5b8d-4aef-97b5-1be000f7afc9 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:52.128772 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcfe6d7-6722-4dc8-8758-c282e9b6fce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:52.129400 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcfe6d7-6722-4dc8-8758-c282e9b6fce1 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:52.129682 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcfe6d7-6722-4dc8-8758-c282e9b6fce1 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:52.129955 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcfe6d7-6722-4dc8-8758-c282e9b6fce1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:52.130747 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcfe6d7-6722-4dc8-8758-c282e9b6fce1 None admin] RBAC: Adding query filter params (group.id=ba5e1c67c05e4d9582187eb89cd1acb5, role.id=8feebc60363c4feb870718b1721e5324, scope.project.id=6c2de9abe1504f6698a9ea2a032c91e4) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:52.131053 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcfe6d7-6722-4dc8-8758-c282e9b6fce1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:52.132536 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcfe6d7-6722-4dc8-8758-c282e9b6fce1 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:52.140311 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 179/371] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Aug 4 21:02:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:52.140923 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 193/372] 10.222.0.25 () {56 vars in 1376 bytes} [Thu Aug 4 21:02:51 2022] GET /identity/v3/role_assignments?group.id=ba5e1c67c05e4d9582187eb89cd1acb5&scope.project.id=6c2de9abe1504f6698a9ea2a032c91e4&role.id=8feebc60363c4feb870718b1721e5324 => generated 260 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:52.141215 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f989ab62-5b8d-4aef-97b5-1be000f7afc9 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:52.141625 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcfe6d7-6722-4dc8-8758-c282e9b6fce1 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:52.167436 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:52.243738 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:52.298736 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f989ab62-5b8d-4aef-97b5-1be000f7afc9 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:52.317971 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcfe6d7-6722-4dc8-8758-c282e9b6fce1 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:52.350422 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8585cc77-03fd-47f3-8542-8e7ff6aae28e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:52.351680 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8585cc77-03fd-47f3-8542-8e7ff6aae28e None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:52.352114 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8585cc77-03fd-47f3-8542-8e7ff6aae28e None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:52.352697 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8585cc77-03fd-47f3-8542-8e7ff6aae28e None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:52.353936 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8585cc77-03fd-47f3-8542-8e7ff6aae28e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:52.354518 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8585cc77-03fd-47f3-8542-8e7ff6aae28e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:52.357300 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8585cc77-03fd-47f3-8542-8e7ff6aae28e None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:52.365991 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 180/373] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Aug 4 21:02:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:52.366431 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79958fc4-289a-492d-ab77-23f189243930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:52.366827 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8585cc77-03fd-47f3-8542-8e7ff6aae28e None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:52.367059 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79958fc4-289a-492d-ab77-23f189243930 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:52.367374 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79958fc4-289a-492d-ab77-23f189243930 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:52.367633 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79958fc4-289a-492d-ab77-23f189243930 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:52.368422 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79958fc4-289a-492d-ab77-23f189243930 None admin] RBAC: Adding query filter params (group.id=ba5e1c67c05e4d9582187eb89cd1acb5, role.id=70587326d8584316b73b380df2ae8e67, scope.project.id=6c2de9abe1504f6698a9ea2a032c91e4) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:52.368731 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79958fc4-289a-492d-ab77-23f189243930 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:52.370163 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79958fc4-289a-492d-ab77-23f189243930 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:52.376516 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 194/374] 10.222.0.25 () {56 vars in 1376 bytes} [Thu Aug 4 21:02:52 2022] GET /identity/v3/role_assignments?group.id=ba5e1c67c05e4d9582187eb89cd1acb5&scope.project.id=6c2de9abe1504f6698a9ea2a032c91e4&role.id=70587326d8584316b73b380df2ae8e67 => generated 260 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:52.389096 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79958fc4-289a-492d-ab77-23f189243930 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:52.463206 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:52.489550 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8585cc77-03fd-47f3-8542-8e7ff6aae28e None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:52.562707 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5293f4f-230c-4e2d-ab6b-5f4b3d9f351c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:52.563403 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5293f4f-230c-4e2d-ab6b-5f4b3d9f351c None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:52.563687 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5293f4f-230c-4e2d-ab6b-5f4b3d9f351c None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:52.563967 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5293f4f-230c-4e2d-ab6b-5f4b3d9f351c None admin] PATH_INFO: `/v3/users/bb0b5e8554204087a31b901e2195eb1f` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:52.565301 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5293f4f-230c-4e2d-ab6b-5f4b3d9f351c None admin] RBAC: Authorizing `identity:get_user(user_id=bb0b5e8554204087a31b901e2195eb1f)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:52.567771 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5293f4f-230c-4e2d-ab6b-5f4b3d9f351c None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:52.569529 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 181/375] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Aug 4 21:02:52 2022] GET /identity/v3/users/bb0b5e8554204087a31b901e2195eb1f => generated 293 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:52.572995 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2934bca2-0ae9-43ee-a919-9927af38f1f3 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:52.573282 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2934bca2-0ae9-43ee-a919-9927af38f1f3 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:52.573556 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2934bca2-0ae9-43ee-a919-9927af38f1f3 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:52.574965 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 182/376] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:02:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:02:52.584309 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2934bca2-0ae9-43ee-a919-9927af38f1f3 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:52.614570 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79958fc4-289a-492d-ab77-23f189243930 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:52.631487 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:52.675609 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920fdf59-716e-47fb-9ed6-dcf3b2a4167d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:52.676249 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920fdf59-716e-47fb-9ed6-dcf3b2a4167d None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:52.676534 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920fdf59-716e-47fb-9ed6-dcf3b2a4167d None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:52.676812 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920fdf59-716e-47fb-9ed6-dcf3b2a4167d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:52.772421 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2934bca2-0ae9-43ee-a919-9927af38f1f3 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:52.815546 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920fdf59-716e-47fb-9ed6-dcf3b2a4167d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:52.817021 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920fdf59-716e-47fb-9ed6-dcf3b2a4167d None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:52.836375 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a8f337-46d6-47b5-90c9-59be2fd70b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:52.837051 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a8f337-46d6-47b5-90c9-59be2fd70b36 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:52.837331 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a8f337-46d6-47b5-90c9-59be2fd70b36 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:52.837600 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a8f337-46d6-47b5-90c9-59be2fd70b36 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:52.838391 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a8f337-46d6-47b5-90c9-59be2fd70b36 None admin] RBAC: Adding query filter params (role.id=5ab0887d24bc41b882f6786b7aab0fe8, user.id=bb0b5e8554204087a31b901e2195eb1f, scope.system=all) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:52.838697 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a8f337-46d6-47b5-90c9-59be2fd70b36 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:52.840186 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a8f337-46d6-47b5-90c9-59be2fd70b36 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:52.868002 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 183/377] 10.222.0.25 () {56 vars in 1308 bytes} [Thu Aug 4 21:02:52 2022] GET /identity/v3/role_assignments?user.id=bb0b5e8554204087a31b901e2195eb1f&scope.system=all&role.id=5ab0887d24bc41b882f6786b7aab0fe8 => generated 510 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:52.868002 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde6d85c-1404-4997-be8a-446b0ac3407d None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:52.868002 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde6d85c-1404-4997-be8a-446b0ac3407d None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:52.868486 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde6d85c-1404-4997-be8a-446b0ac3407d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:52.885783 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:52.925194 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-dde6d85c-1404-4997-be8a-446b0ac3407d None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:52.926404 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-dde6d85c-1404-4997-be8a-446b0ac3407d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:53.082791 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 184/378] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:02:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:02:53.108238 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 195/379] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Aug 4 21:02:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 720 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 21:02:53.114769 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde6d85c-1404-4997-be8a-446b0ac3407d None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:53.121773 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920fdf59-716e-47fb-9ed6-dcf3b2a4167d None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:53.149478 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:53.304320 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde6d85c-1404-4997-be8a-446b0ac3407d None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:53.324060 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920fdf59-716e-47fb-9ed6-dcf3b2a4167d None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:53.347705 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:53.361488 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f82cb85-ee40-48ec-a5dc-614e5017c1f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:53.362856 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f82cb85-ee40-48ec-a5dc-614e5017c1f4 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:53.363135 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f82cb85-ee40-48ec-a5dc-614e5017c1f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:53.363425 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f82cb85-ee40-48ec-a5dc-614e5017c1f4 None admin] PATH_INFO: `/v3/projects/6da47929012341928c673a2e0d1e97f0` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:53.366004 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f82cb85-ee40-48ec-a5dc-614e5017c1f4 None admin] RBAC: Authorizing `identity:get_project(project_id=6da47929012341928c673a2e0d1e97f0)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:53.367638 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f82cb85-ee40-48ec-a5dc-614e5017c1f4 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:53.369604 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 185/380] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Aug 4 21:02:53 2022] GET /identity/v3/projects/6da47929012341928c673a2e0d1e97f0 => generated 344 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:53.373446 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28040fe0-a1d8-41fe-81a5-b9f2712aedfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:53.374188 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28040fe0-a1d8-41fe-81a5-b9f2712aedfe None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:53.374479 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28040fe0-a1d8-41fe-81a5-b9f2712aedfe None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:53.374756 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28040fe0-a1d8-41fe-81a5-b9f2712aedfe None admin] PATH_INFO: `/v3/roles/member` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:53.375093 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f82cb85-ee40-48ec-a5dc-614e5017c1f4 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:53.388497 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28040fe0-a1d8-41fe-81a5-b9f2712aedfe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:02:53.394076 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-28040fe0-a1d8-41fe-81a5-b9f2712aedfe None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 21:02:53.394076 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:02:53.394076 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:02:53.394076 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:02:53.394076 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:02:53.394076 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:02:53.394076 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:02:53.394076 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:02:53.394076 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:02:53.394076 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:02:53.394076 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:02:53.394076 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:02:53.394076 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:02:53.394076 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:02:53.394076 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:02:53.394076 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:02:53.394952 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:02:53.394952 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:02:53.394952 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:02:53.394952 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:02:53.394952 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:02:53.394952 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:02:53.394952 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:02:53.394952 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:02:53.394952 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:02:53.394952 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:02:53.394952 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:02:53.394952 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:02:53.394952 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:02:53.394952 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:02:53.394952 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:02:53.394952 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:02:53.395792 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:02:53.395792 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:02:53.395792 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:02:53.395792 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:02:53.395792 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:02:53.395792 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:02:53.395792 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:02:53.395792 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:02:53.395792 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:02:53.395792 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:02:53.395792 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 21:02:53.395792 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:02:53.395792 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 196/381] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Aug 4 21:02:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:02:53.402498 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28040fe0-a1d8-41fe-81a5-b9f2712aedfe None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:53.530453 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:53.546175 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f82cb85-ee40-48ec-a5dc-614e5017c1f4 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:53.584801 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b66fbd2-f54e-462c-9e9a-474c11920d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:53.585389 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28040fe0-a1d8-41fe-81a5-b9f2712aedfe None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:53.585861 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b66fbd2-f54e-462c-9e9a-474c11920d72 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:53.586379 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b66fbd2-f54e-462c-9e9a-474c11920d72 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:53.586901 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b66fbd2-f54e-462c-9e9a-474c11920d72 None admin] PATH_INFO: `/v3/groups/0117a08ce6764930bb139519a935554d` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:53.593392 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b66fbd2-f54e-462c-9e9a-474c11920d72 None admin] RBAC: Authorizing `identity:get_group(group_id=0117a08ce6764930bb139519a935554d)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:53.595138 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b66fbd2-f54e-462c-9e9a-474c11920d72 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:53.598330 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 186/382] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Aug 4 21:02:53 2022] GET /identity/v3/groups/0117a08ce6764930bb139519a935554d => generated 229 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:02:53.603780 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b66fbd2-f54e-462c-9e9a-474c11920d72 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:53.632493 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4408cbb6-f42e-43de-99e3-e04f9f39a03f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:53.633236 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4408cbb6-f42e-43de-99e3-e04f9f39a03f None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:53.633519 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4408cbb6-f42e-43de-99e3-e04f9f39a03f None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:53.633792 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4408cbb6-f42e-43de-99e3-e04f9f39a03f None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:53.635417 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4408cbb6-f42e-43de-99e3-e04f9f39a03f None admin] RBAC: Adding query filter params (name=member) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:53.635731 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4408cbb6-f42e-43de-99e3-e04f9f39a03f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:53.640281 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4408cbb6-f42e-43de-99e3-e04f9f39a03f None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:53.655993 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 197/383] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Aug 4 21:02:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:53.662109 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4408cbb6-f42e-43de-99e3-e04f9f39a03f None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:53.720813 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:53.775511 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b66fbd2-f54e-462c-9e9a-474c11920d72 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:53.815402 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9404cb-5e84-4dc7-bb96-59654fe94366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:53.817437 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9404cb-5e84-4dc7-bb96-59654fe94366 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:53.817929 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9404cb-5e84-4dc7-bb96-59654fe94366 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:53.818380 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9404cb-5e84-4dc7-bb96-59654fe94366 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:53.819503 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9404cb-5e84-4dc7-bb96-59654fe94366 None admin] RBAC: Adding query filter params (group.id=0117a08ce6764930bb139519a935554d, role.id=a432dd4735944ecfab2fa5ce1705d356, scope.project.id=6da47929012341928c673a2e0d1e97f0) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:53.819995 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9404cb-5e84-4dc7-bb96-59654fe94366 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:53.821651 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9404cb-5e84-4dc7-bb96-59654fe94366 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:53.829262 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 187/384] 10.222.0.25 () {56 vars in 1376 bytes} [Thu Aug 4 21:02:53 2022] GET /identity/v3/role_assignments?group.id=0117a08ce6764930bb139519a935554d&scope.project.id=6da47929012341928c673a2e0d1e97f0&role.id=a432dd4735944ecfab2fa5ce1705d356 => generated 260 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:53.877047 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4408cbb6-f42e-43de-99e3-e04f9f39a03f None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:53.929927 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bc01e6-5f55-4f29-9c5a-9e57508d431f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:53.930587 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bc01e6-5f55-4f29-9c5a-9e57508d431f None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:53.930873 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bc01e6-5f55-4f29-9c5a-9e57508d431f None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:53.931151 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bc01e6-5f55-4f29-9c5a-9e57508d431f None admin] PATH_INFO: `/v3/groups/ba5e1c67c05e4d9582187eb89cd1acb5` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:53.936224 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bc01e6-5f55-4f29-9c5a-9e57508d431f None admin] RBAC: Authorizing `identity:get_group(group_id=ba5e1c67c05e4d9582187eb89cd1acb5)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:53.937738 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bc01e6-5f55-4f29-9c5a-9e57508d431f None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:53.939903 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 198/385] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Aug 4 21:02:53 2022] GET /identity/v3/groups/ba5e1c67c05e4d9582187eb89cd1acb5 => generated 226 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:53.945885 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9404cb-5e84-4dc7-bb96-59654fe94366 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:54.071968 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9404cb-5e84-4dc7-bb96-59654fe94366 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:54.099840 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:54.108787 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc2d610-aa0e-452e-a80d-ee0d074ae5b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:54.109690 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc2d610-aa0e-452e-a80d-ee0d074ae5b7 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:54.110131 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc2d610-aa0e-452e-a80d-ee0d074ae5b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:54.110561 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc2d610-aa0e-452e-a80d-ee0d074ae5b7 None admin] PATH_INFO: `/v3/projects/b8f804a236884f039371233cce01ecef` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:54.119404 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc2d610-aa0e-452e-a80d-ee0d074ae5b7 None admin] RBAC: Authorizing `identity:get_project(project_id=b8f804a236884f039371233cce01ecef)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:54.119404 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc2d610-aa0e-452e-a80d-ee0d074ae5b7 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:54.119404 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 188/386] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Aug 4 21:02:53 2022] GET /identity/v3/projects/b8f804a236884f039371233cce01ecef => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:54.124351 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bc01e6-5f55-4f29-9c5a-9e57508d431f None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:54.127484 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:54.309954 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bc01e6-5f55-4f29-9c5a-9e57508d431f None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:54.359040 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4031db6a-6a9f-4379-bdf9-dabc9752ae87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:54.359806 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4031db6a-6a9f-4379-bdf9-dabc9752ae87 None admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:54.360024 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4031db6a-6a9f-4379-bdf9-dabc9752ae87 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:54.360299 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4031db6a-6a9f-4379-bdf9-dabc9752ae87 None admin] PATH_INFO: `/v3/projects/b8f804a236884f039371233cce01ecef/groups/ba5e1c67c05e4d9582187eb89cd1acb5/roles/8feebc60363c4feb870718b1721e5324` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:54.362821 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4031db6a-6a9f-4379-bdf9-dabc9752ae87 None admin] RBAC: Authorizing `identity:create_grant(project_id=b8f804a236884f039371233cce01ecef, group_id=ba5e1c67c05e4d9582187eb89cd1acb5, role_id=8feebc60363c4feb870718b1721e5324)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:54.364283 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4031db6a-6a9f-4379-bdf9-dabc9752ae87 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:54.400721 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d950640-8de7-48c6-ac72-e8278d4e9685 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:54.401379 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d950640-8de7-48c6-ac72-e8278d4e9685 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:54.401856 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d950640-8de7-48c6-ac72-e8278d4e9685 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:54.403759 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 189/387] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:02:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:02:54.410362 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ff7211-4f72-4f6c-a0a4-9d125d714580 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:54.410661 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ff7211-4f72-4f6c-a0a4-9d125d714580 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:54.410944 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ff7211-4f72-4f6c-a0a4-9d125d714580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:54.433620 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 199/388] 10.222.0.25 () {58 vars in 1442 bytes} [Thu Aug 4 21:02:54 2022] PUT /identity/v3/projects/b8f804a236884f039371233cce01ecef/groups/ba5e1c67c05e4d9582187eb89cd1acb5/roles/8feebc60363c4feb870718b1721e5324 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:02:54.437029 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-b0ff7211-4f72-4f6c-a0a4-9d125d714580 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:54.438248 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-b0ff7211-4f72-4f6c-a0a4-9d125d714580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:54.523483 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 190/389] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:02:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:02:54.557507 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4031db6a-6a9f-4379-bdf9-dabc9752ae87 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:54.639003 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:54.692704 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4031db6a-6a9f-4379-bdf9-dabc9752ae87 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:54.729417 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b20642-131b-4c53-87e5-994047b7432f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:54.730402 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b20642-131b-4c53-87e5-994047b7432f None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:54.730897 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b20642-131b-4c53-87e5-994047b7432f None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:54.731403 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b20642-131b-4c53-87e5-994047b7432f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:54.742214 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b20642-131b-4c53-87e5-994047b7432f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:02:54.748084 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-39b20642-131b-4c53-87e5-994047b7432f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 21:02:54.748084 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:02:54.748084 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:02:54.748084 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:02:54.748084 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:02:54.748084 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:02:54.748084 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:02:54.748084 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:02:54.748084 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:02:54.748084 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:02:54.748084 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:02:54.748084 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:02:54.748084 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:02:54.748084 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:02:54.748084 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:02:54.748084 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:02:54.748970 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:02:54.748970 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:02:54.748970 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:02:54.748970 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:02:54.748970 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:02:54.748970 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:02:54.748970 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:02:54.748970 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:02:54.748970 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:02:54.748970 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:02:54.748970 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:02:54.748970 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:02:54.748970 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:02:54.748970 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:02:54.748970 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:02:54.748970 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:02:54.749753 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:02:54.749753 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:02:54.749753 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:02:54.749753 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:02:54.749753 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:02:54.749753 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:02:54.749753 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:02:54.749753 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:02:54.749753 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:02:54.749753 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:02:54.749753 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 21:02:54.749753 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:02:54.752389 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 200/390] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Aug 4 21:02:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:02:54.758408 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ff7211-4f72-4f6c-a0a4-9d125d714580 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:54.893377 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ff7211-4f72-4f6c-a0a4-9d125d714580 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:54.919817 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4645f7fa-9a8e-48af-badd-be53e19c3b00 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:54.920186 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4645f7fa-9a8e-48af-badd-be53e19c3b00 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:54.920368 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4645f7fa-9a8e-48af-badd-be53e19c3b00 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:54.925422 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 201/391] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:02:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:02:54.931774 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:54.932669 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4836414b-0a34-4cb3-80c1-78ce602f25b6 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:54.932944 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4836414b-0a34-4cb3-80c1-78ce602f25b6 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:54.933220 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4836414b-0a34-4cb3-80c1-78ce602f25b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:54.941838 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffac09d-2707-4fbd-8c48-da867a7f3368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:54.942819 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffac09d-2707-4fbd-8c48-da867a7f3368 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:54.943293 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffac09d-2707-4fbd-8c48-da867a7f3368 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:54.943898 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffac09d-2707-4fbd-8c48-da867a7f3368 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:54.950638 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffac09d-2707-4fbd-8c48-da867a7f3368 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:54.950638 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffac09d-2707-4fbd-8c48-da867a7f3368 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:54.950638 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffac09d-2707-4fbd-8c48-da867a7f3368 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:54.952982 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-4836414b-0a34-4cb3-80c1-78ce602f25b6 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:54.954189 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-4836414b-0a34-4cb3-80c1-78ce602f25b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:54.956872 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 191/392] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Aug 4 21:02:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:54.958392 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb282dcd-0311-4f88-9fb8-cc418d6dc631 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:54.958875 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb282dcd-0311-4f88-9fb8-cc418d6dc631 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:54.959361 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb282dcd-0311-4f88-9fb8-cc418d6dc631 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:54.961046 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 192/393] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:02:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:02:54.961944 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb282dcd-0311-4f88-9fb8-cc418d6dc631 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:55.038091 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 202/394] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:02:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:02:55.045393 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47639a10-6b7f-4d5c-badb-3cf90302bcd4 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:55.045918 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47639a10-6b7f-4d5c-badb-3cf90302bcd4 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:55.046433 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47639a10-6b7f-4d5c-badb-3cf90302bcd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:55.078989 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-47639a10-6b7f-4d5c-badb-3cf90302bcd4 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:55.080347 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb282dcd-0311-4f88-9fb8-cc418d6dc631 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:55.080778 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-47639a10-6b7f-4d5c-badb-3cf90302bcd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:55.120975 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aad0444-551c-401d-a38e-3f7111c9d717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:55.121649 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aad0444-551c-401d-a38e-3f7111c9d717 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:55.121929 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aad0444-551c-401d-a38e-3f7111c9d717 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:55.122284 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aad0444-551c-401d-a38e-3f7111c9d717 None admin] PATH_INFO: `/v3/groups/ba5e1c67c05e4d9582187eb89cd1acb5` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:55.123528 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aad0444-551c-401d-a38e-3f7111c9d717 None admin] RBAC: Authorizing `identity:get_group(group_id=ba5e1c67c05e4d9582187eb89cd1acb5)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:55.125146 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aad0444-551c-401d-a38e-3f7111c9d717 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:55.127883 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 193/395] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Aug 4 21:02:54 2022] GET /identity/v3/groups/ba5e1c67c05e4d9582187eb89cd1acb5 => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:55.128657 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aad0444-551c-401d-a38e-3f7111c9d717 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:55.140173 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:55.164449 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 203/396] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:02:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:02:55.165768 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47639a10-6b7f-4d5c-badb-3cf90302bcd4 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:55.226928 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:55.272465 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aad0444-551c-401d-a38e-3f7111c9d717 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:55.309126 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7115f8f-15e9-458a-a747-af516c699e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:55.310000 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7115f8f-15e9-458a-a747-af516c699e55 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:55.310280 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7115f8f-15e9-458a-a747-af516c699e55 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:55.310595 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7115f8f-15e9-458a-a747-af516c699e55 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:55.313741 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47639a10-6b7f-4d5c-badb-3cf90302bcd4 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:55.321737 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7115f8f-15e9-458a-a747-af516c699e55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:02:55.327636 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-f7115f8f-15e9-458a-a747-af516c699e55 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 21:02:55.327636 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:02:55.327636 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:02:55.327636 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:02:55.327636 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:02:55.327636 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:02:55.327636 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:02:55.327636 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:02:55.327636 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:02:55.327636 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:02:55.327636 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:02:55.327636 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:02:55.327636 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:02:55.327636 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:02:55.327636 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:02:55.327636 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:02:55.328529 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:02:55.328529 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:02:55.328529 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:02:55.328529 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:02:55.328529 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:02:55.328529 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:02:55.328529 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:02:55.328529 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:02:55.328529 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:02:55.328529 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:02:55.328529 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:02:55.328529 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:02:55.328529 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:02:55.328529 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:02:55.328529 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:02:55.328529 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:02:55.329324 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:02:55.329324 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:02:55.329324 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:02:55.329324 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:02:55.329324 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:02:55.329324 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:02:55.329324 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:02:55.329324 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:02:55.329324 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:02:55.329324 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:02:55.329324 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 21:02:55.329324 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:02:55.329324 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 194/397] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Aug 4 21:02:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:02:55.329967 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7115f8f-15e9-458a-a747-af516c699e55 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:55.369157 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa425643-51b8-4f11-8eb3-28a16f2d9a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:55.369861 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa425643-51b8-4f11-8eb3-28a16f2d9a17 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:55.370333 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa425643-51b8-4f11-8eb3-28a16f2d9a17 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:55.370437 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa425643-51b8-4f11-8eb3-28a16f2d9a17 None admin] PATH_INFO: `/v3/projects/b8f804a236884f039371233cce01ecef` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:55.371625 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa425643-51b8-4f11-8eb3-28a16f2d9a17 None admin] RBAC: Authorizing `identity:get_project(project_id=b8f804a236884f039371233cce01ecef)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:55.373080 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa425643-51b8-4f11-8eb3-28a16f2d9a17 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:55.375671 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 204/398] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Aug 4 21:02:55 2022] GET /identity/v3/projects/b8f804a236884f039371233cce01ecef => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:55.375802 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa425643-51b8-4f11-8eb3-28a16f2d9a17 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:55.438949 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:55.455581 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7115f8f-15e9-458a-a747-af516c699e55 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:55.476396 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:55.499243 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9741c7-a368-4556-907d-d43b8ec57245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:55.500364 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9741c7-a368-4556-907d-d43b8ec57245 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:55.500654 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9741c7-a368-4556-907d-d43b8ec57245 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:55.500930 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9741c7-a368-4556-907d-d43b8ec57245 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:55.511212 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa425643-51b8-4f11-8eb3-28a16f2d9a17 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:55.515747 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9741c7-a368-4556-907d-d43b8ec57245 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:02:55.522177 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-4c9741c7-a368-4556-907d-d43b8ec57245 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 21:02:55.522177 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:02:55.522177 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:02:55.522177 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:02:55.522177 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:02:55.522177 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:02:55.522177 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:02:55.522177 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:02:55.522177 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:02:55.522177 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:02:55.522177 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:02:55.522177 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:02:55.522177 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:02:55.522177 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:02:55.522177 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:02:55.522177 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:02:55.523064 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:02:55.523064 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:02:55.523064 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:02:55.523064 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:02:55.523064 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:02:55.523064 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:02:55.523064 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:02:55.523064 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:02:55.523064 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:02:55.523064 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:02:55.523064 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:02:55.523064 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:02:55.523064 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:02:55.523064 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:02:55.523064 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:02:55.523064 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:02:55.523901 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:02:55.523901 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:02:55.523901 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:02:55.523901 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:02:55.523901 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:02:55.523901 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:02:55.523901 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:02:55.523901 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:02:55.523901 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:02:55.523901 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:02:55.523901 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 21:02:55.523901 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:02:55.523901 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 195/399] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Aug 4 21:02:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:02:55.524555 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9741c7-a368-4556-907d-d43b8ec57245 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:55.556494 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6166fa6b-db7d-454c-9c13-abad1a7e988a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:55.557786 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6166fa6b-db7d-454c-9c13-abad1a7e988a None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:55.558516 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6166fa6b-db7d-454c-9c13-abad1a7e988a None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:55.558985 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6166fa6b-db7d-454c-9c13-abad1a7e988a None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:55.560555 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6166fa6b-db7d-454c-9c13-abad1a7e988a None admin] RBAC: Adding query filter params (name=member) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:55.561451 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6166fa6b-db7d-454c-9c13-abad1a7e988a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:55.563514 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6166fa6b-db7d-454c-9c13-abad1a7e988a None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:55.578821 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 205/400] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Aug 4 21:02:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:55.578821 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6166fa6b-db7d-454c-9c13-abad1a7e988a None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:55.644744 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9741c7-a368-4556-907d-d43b8ec57245 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:55.680770 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bc8bc0-d569-4f81-ac95-69f4ecba060b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:55.681742 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bc8bc0-d569-4f81-ac95-69f4ecba060b None admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:55.682222 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bc8bc0-d569-4f81-ac95-69f4ecba060b None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:55.682735 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bc8bc0-d569-4f81-ac95-69f4ecba060b None admin] PATH_INFO: `/v3/projects/b8f804a236884f039371233cce01ecef/groups/ba5e1c67c05e4d9582187eb89cd1acb5/roles/70587326d8584316b73b380df2ae8e67` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:55.685327 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bc8bc0-d569-4f81-ac95-69f4ecba060b None admin] RBAC: Authorizing `identity:create_grant(project_id=b8f804a236884f039371233cce01ecef, group_id=ba5e1c67c05e4d9582187eb89cd1acb5, role_id=70587326d8584316b73b380df2ae8e67)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:55.687017 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bc8bc0-d569-4f81-ac95-69f4ecba060b None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:55.705797 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6166fa6b-db7d-454c-9c13-abad1a7e988a None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:55.719518 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:55.741113 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81415325-2612-40de-b6f3-4209fb7de20a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:55.741745 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81415325-2612-40de-b6f3-4209fb7de20a None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:55.742022 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81415325-2612-40de-b6f3-4209fb7de20a None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:55.742290 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81415325-2612-40de-b6f3-4209fb7de20a None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:55.742998 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81415325-2612-40de-b6f3-4209fb7de20a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:55.743299 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81415325-2612-40de-b6f3-4209fb7de20a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:55.744775 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81415325-2612-40de-b6f3-4209fb7de20a None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:55.760032 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 206/401] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Aug 4 21:02:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:55.760032 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81415325-2612-40de-b6f3-4209fb7de20a None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:55.772722 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 196/402] 10.222.0.25 () {58 vars in 1442 bytes} [Thu Aug 4 21:02:55 2022] PUT /identity/v3/projects/b8f804a236884f039371233cce01ecef/groups/ba5e1c67c05e4d9582187eb89cd1acb5/roles/70587326d8584316b73b380df2ae8e67 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:02:55.773368 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bc8bc0-d569-4f81-ac95-69f4ecba060b None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:55.842139 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:55.961112 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bc8bc0-d569-4f81-ac95-69f4ecba060b None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:55.993299 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81415325-2612-40de-b6f3-4209fb7de20a None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:56.039615 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a670023-4374-4b5e-a0a1-777e1f2e2f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:56.040448 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a670023-4374-4b5e-a0a1-777e1f2e2f18 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:56.040578 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a670023-4374-4b5e-a0a1-777e1f2e2f18 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:56.040833 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a670023-4374-4b5e-a0a1-777e1f2e2f18 None admin] PATH_INFO: `/v3/groups/ba5e1c67c05e4d9582187eb89cd1acb5` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:56.042036 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a670023-4374-4b5e-a0a1-777e1f2e2f18 None admin] RBAC: Authorizing `identity:get_group(group_id=ba5e1c67c05e4d9582187eb89cd1acb5)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:56.042531 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dda4fce-42e3-406a-a58c-f54825f873e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:56.043496 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dda4fce-42e3-406a-a58c-f54825f873e8 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:56.043660 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a670023-4374-4b5e-a0a1-777e1f2e2f18 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:56.044114 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dda4fce-42e3-406a-a58c-f54825f873e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:56.044496 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dda4fce-42e3-406a-a58c-f54825f873e8 None admin] PATH_INFO: `/v3/groups/ba5e1c67c05e4d9582187eb89cd1acb5` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:56.045648 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 197/403] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Aug 4 21:02:55 2022] GET /identity/v3/groups/ba5e1c67c05e4d9582187eb89cd1acb5 => generated 226 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:56.045830 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dda4fce-42e3-406a-a58c-f54825f873e8 None admin] RBAC: Authorizing `identity:get_group(group_id=ba5e1c67c05e4d9582187eb89cd1acb5)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:56.048361 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dda4fce-42e3-406a-a58c-f54825f873e8 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:56.050549 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 207/404] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Aug 4 21:02:55 2022] GET /identity/v3/groups/ba5e1c67c05e4d9582187eb89cd1acb5 => generated 226 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:56.051859 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a670023-4374-4b5e-a0a1-777e1f2e2f18 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:56.056038 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dda4fce-42e3-406a-a58c-f54825f873e8 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:56.119089 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:56.179900 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a670023-4374-4b5e-a0a1-777e1f2e2f18 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:56.182671 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:56.214810 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dda4fce-42e3-406a-a58c-f54825f873e8 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:56.233679 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b6bfac-fb79-4262-a24f-86380c669ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:56.234534 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b6bfac-fb79-4262-a24f-86380c669ae0 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:56.234973 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b6bfac-fb79-4262-a24f-86380c669ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:56.235686 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b6bfac-fb79-4262-a24f-86380c669ae0 None admin] PATH_INFO: `/v3/projects/6c2de9abe1504f6698a9ea2a032c91e4` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:56.237657 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b6bfac-fb79-4262-a24f-86380c669ae0 None admin] RBAC: Authorizing `identity:get_project(project_id=6c2de9abe1504f6698a9ea2a032c91e4)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:56.239424 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b6bfac-fb79-4262-a24f-86380c669ae0 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:56.241859 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 198/405] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Aug 4 21:02:56 2022] GET /identity/v3/projects/6c2de9abe1504f6698a9ea2a032c91e4 => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:56.250634 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b6bfac-fb79-4262-a24f-86380c669ae0 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:56.308830 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbee2976-deff-4180-b602-cca5d9a801ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:56.309985 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbee2976-deff-4180-b602-cca5d9a801ef None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:56.310599 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbee2976-deff-4180-b602-cca5d9a801ef None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:56.311012 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbee2976-deff-4180-b602-cca5d9a801ef None admin] PATH_INFO: `/v3/projects/6c2de9abe1504f6698a9ea2a032c91e4` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:56.321448 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbee2976-deff-4180-b602-cca5d9a801ef None admin] RBAC: Authorizing `identity:get_project(project_id=6c2de9abe1504f6698a9ea2a032c91e4)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:56.326115 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbee2976-deff-4180-b602-cca5d9a801ef None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:56.339401 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 208/406] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Aug 4 21:02:56 2022] GET /identity/v3/projects/6c2de9abe1504f6698a9ea2a032c91e4 => generated 302 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:02:56.346007 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbee2976-deff-4180-b602-cca5d9a801ef None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:56.375994 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b6bfac-fb79-4262-a24f-86380c669ae0 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:56.433939 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d23283f-dc3e-4930-b146-d5575ae856f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:56.434988 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d23283f-dc3e-4930-b146-d5575ae856f1 None admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:56.435276 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d23283f-dc3e-4930-b146-d5575ae856f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:56.435584 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d23283f-dc3e-4930-b146-d5575ae856f1 None admin] PATH_INFO: `/v3/projects/6c2de9abe1504f6698a9ea2a032c91e4/groups/ba5e1c67c05e4d9582187eb89cd1acb5/roles/70587326d8584316b73b380df2ae8e67` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:56.437816 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d23283f-dc3e-4930-b146-d5575ae856f1 None admin] RBAC: Authorizing `identity:create_grant(project_id=6c2de9abe1504f6698a9ea2a032c91e4, group_id=ba5e1c67c05e4d9582187eb89cd1acb5, role_id=70587326d8584316b73b380df2ae8e67)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:56.439286 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d23283f-dc3e-4930-b146-d5575ae856f1 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:56.518963 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbee2976-deff-4180-b602-cca5d9a801ef None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:56.526022 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 199/407] 10.222.0.25 () {58 vars in 1442 bytes} [Thu Aug 4 21:02:56 2022] PUT /identity/v3/projects/6c2de9abe1504f6698a9ea2a032c91e4/groups/ba5e1c67c05e4d9582187eb89cd1acb5/roles/70587326d8584316b73b380df2ae8e67 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:02:56.545086 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:56.574544 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf4876e-0584-4690-bb35-9d427eebcd70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:56.575231 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf4876e-0584-4690-bb35-9d427eebcd70 None admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:56.575548 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf4876e-0584-4690-bb35-9d427eebcd70 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:56.575823 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf4876e-0584-4690-bb35-9d427eebcd70 None admin] PATH_INFO: `/v3/projects/6c2de9abe1504f6698a9ea2a032c91e4/groups/ba5e1c67c05e4d9582187eb89cd1acb5/roles/8feebc60363c4feb870718b1721e5324` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:56.577989 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf4876e-0584-4690-bb35-9d427eebcd70 None admin] RBAC: Authorizing `identity:create_grant(project_id=6c2de9abe1504f6698a9ea2a032c91e4, group_id=ba5e1c67c05e4d9582187eb89cd1acb5, role_id=8feebc60363c4feb870718b1721e5324)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:56.579457 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf4876e-0584-4690-bb35-9d427eebcd70 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:56.597430 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f276178d-bf9d-44b1-83c1-0599628d3e6a None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:56.597776 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f276178d-bf9d-44b1-83c1-0599628d3e6a None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:56.597982 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f276178d-bf9d-44b1-83c1-0599628d3e6a None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:56.600080 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 200/408] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:02:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 21:02:56.605885 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1d65c1-3045-4578-8433-f9635a5a3dbc None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:56.606211 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1d65c1-3045-4578-8433-f9635a5a3dbc None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:56.606441 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1d65c1-3045-4578-8433-f9635a5a3dbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:56.627716 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-da1d65c1-3045-4578-8433-f9635a5a3dbc None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:56.629058 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-da1d65c1-3045-4578-8433-f9635a5a3dbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:56.655633 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 209/409] 10.222.0.25 () {58 vars in 1442 bytes} [Thu Aug 4 21:02:56 2022] PUT /identity/v3/projects/6c2de9abe1504f6698a9ea2a032c91e4/groups/ba5e1c67c05e4d9582187eb89cd1acb5/roles/8feebc60363c4feb870718b1721e5324 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:02:56.664024 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:56.718388 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 201/410] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:02:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 21:02:56.757002 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf4876e-0584-4690-bb35-9d427eebcd70 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:56.877883 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8686b7f7-2dc9-4383-a15d-6e5c8619a48c None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:56.878829 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8686b7f7-2dc9-4383-a15d-6e5c8619a48c None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:56.879306 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8686b7f7-2dc9-4383-a15d-6e5c8619a48c None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:56.882932 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 202/411] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:02:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:02:56.887206 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e380693-2065-4c4d-b801-cf2ea0ac87f6 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:56.887783 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e380693-2065-4c4d-b801-cf2ea0ac87f6 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:56.888314 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e380693-2065-4c4d-b801-cf2ea0ac87f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:56.906260 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf4876e-0584-4690-bb35-9d427eebcd70 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:56.913404 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-1e380693-2065-4c4d-b801-cf2ea0ac87f6 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:56.915545 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-1e380693-2065-4c4d-b801-cf2ea0ac87f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:56.949253 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b0f2fc-67f4-4f6c-8ed3-a103308f7773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:56.949916 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b0f2fc-67f4-4f6c-8ed3-a103308f7773 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:56.950246 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b0f2fc-67f4-4f6c-8ed3-a103308f7773 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:56.950527 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b0f2fc-67f4-4f6c-8ed3-a103308f7773 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:56.966573 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b0f2fc-67f4-4f6c-8ed3-a103308f7773 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:02:56.974589 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-e7b0f2fc-67f4-4f6c-8ed3-a103308f7773 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:02:56.974589 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:02:56.974589 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:02:56.974589 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:02:56.974589 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:02:56.974589 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:02:56.974589 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:02:56.974589 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:02:56.974589 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:02:56.974589 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:02:56.974589 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:02:56.974589 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:02:56.974589 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:02:56.974589 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:02:56.974589 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:02:56.974589 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:02:56.979454 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:02:56.979454 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:02:56.979454 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:02:56.979454 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:02:56.979454 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:02:56.979454 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:02:56.979454 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:02:56.979454 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:02:56.979454 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:02:56.979454 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:02:56.979454 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:02:56.979454 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:02:56.979454 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:02:56.979454 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:02:56.979454 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:02:56.979454 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:02:56.980375 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:02:56.980375 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:02:56.980375 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:02:56.980375 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:02:56.980375 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:02:56.980375 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:02:56.980375 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:02:56.980375 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:02:56.980375 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:02:56.980375 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:02:56.980375 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:02:56.980375 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:02:56.980375 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 210/412] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Aug 4 21:02:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:02:56.985517 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b0f2fc-67f4-4f6c-8ed3-a103308f7773 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:57.024282 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 203/413] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:02:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 21:02:57.035422 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:57.074387 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e380693-2065-4c4d-b801-cf2ea0ac87f6 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:57.158221 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b0f2fc-67f4-4f6c-8ed3-a103308f7773 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:57.193808 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:57.203717 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e380693-2065-4c4d-b801-cf2ea0ac87f6 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:57.236723 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dac93c-0139-4fbf-8cf4-6d23067628c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:57.237946 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dac93c-0139-4fbf-8cf4-6d23067628c7 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:57.238230 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dac93c-0139-4fbf-8cf4-6d23067628c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:57.238507 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dac93c-0139-4fbf-8cf4-6d23067628c7 None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:57.239555 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dac93c-0139-4fbf-8cf4-6d23067628c7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:57.239555 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dac93c-0139-4fbf-8cf4-6d23067628c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:57.240952 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dac93c-0139-4fbf-8cf4-6d23067628c7 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:57.260017 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9561a0-a091-45ff-81f1-009246ccfa79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:57.261281 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9561a0-a091-45ff-81f1-009246ccfa79 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:57.263748 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9561a0-a091-45ff-81f1-009246ccfa79 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:57.264268 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9561a0-a091-45ff-81f1-009246ccfa79 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:57.265255 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 211/414] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Aug 4 21:02:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:57.270289 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dac93c-0139-4fbf-8cf4-6d23067628c7 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:57.288108 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9561a0-a091-45ff-81f1-009246ccfa79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:02:57.295129 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-5e9561a0-a091-45ff-81f1-009246ccfa79 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 21:02:57.295129 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:02:57.295129 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:02:57.295129 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:02:57.295129 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:02:57.295129 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:02:57.295129 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:02:57.295129 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:02:57.295129 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:02:57.295129 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:02:57.295129 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:02:57.295129 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:02:57.295129 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:02:57.295129 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:02:57.295129 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:02:57.295129 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:02:57.296052 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:02:57.296052 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:02:57.296052 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:02:57.296052 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:02:57.296052 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:02:57.296052 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:02:57.296052 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:02:57.296052 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:02:57.296052 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:02:57.296052 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:02:57.296052 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:02:57.296052 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:02:57.296052 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:02:57.296052 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:02:57.296052 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:02:57.296052 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:02:57.296856 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:02:57.296856 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:02:57.296856 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:02:57.296856 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:02:57.296856 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:02:57.296856 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:02:57.296856 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:02:57.296856 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:02:57.296856 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:02:57.296856 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:02:57.296856 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 21:02:57.296856 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:02:57.298010 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 204/415] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Aug 4 21:02:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:02:57.304291 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9561a0-a091-45ff-81f1-009246ccfa79 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:57.429434 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:57.443533 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dac93c-0139-4fbf-8cf4-6d23067628c7 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:57.508950 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9561a0-a091-45ff-81f1-009246ccfa79 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:57.511843 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28a02b7-4931-4683-985b-e09b3db6f9dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:57.512499 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28a02b7-4931-4683-985b-e09b3db6f9dc None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:57.512781 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28a02b7-4931-4683-985b-e09b3db6f9dc None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:57.513057 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28a02b7-4931-4683-985b-e09b3db6f9dc None admin] PATH_INFO: `/v3/groups/0117a08ce6764930bb139519a935554d` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:57.514208 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28a02b7-4931-4683-985b-e09b3db6f9dc None admin] RBAC: Authorizing `identity:get_group(group_id=0117a08ce6764930bb139519a935554d)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:57.515690 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28a02b7-4931-4683-985b-e09b3db6f9dc None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:57.517614 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 212/416] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Aug 4 21:02:57 2022] GET /identity/v3/groups/0117a08ce6764930bb139519a935554d => generated 229 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:57.526428 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28a02b7-4931-4683-985b-e09b3db6f9dc None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:57.561446 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c639a4-1d0f-4a23-923a-c59c7ca5883b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:57.562083 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c639a4-1d0f-4a23-923a-c59c7ca5883b None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:57.562365 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c639a4-1d0f-4a23-923a-c59c7ca5883b None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:57.562638 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c639a4-1d0f-4a23-923a-c59c7ca5883b None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:57.563387 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c639a4-1d0f-4a23-923a-c59c7ca5883b None admin] RBAC: Adding query filter params (name=member) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:57.563694 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c639a4-1d0f-4a23-923a-c59c7ca5883b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:57.565183 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c639a4-1d0f-4a23-923a-c59c7ca5883b None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:57.589730 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 205/417] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Aug 4 21:02:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:57.597923 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c639a4-1d0f-4a23-923a-c59c7ca5883b None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:57.634096 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:57.703842 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28a02b7-4931-4683-985b-e09b3db6f9dc None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:57.772034 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcada5a7-a8d7-4630-82c5-e0ee0aa3880b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:57.772700 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcada5a7-a8d7-4630-82c5-e0ee0aa3880b None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:57.772990 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcada5a7-a8d7-4630-82c5-e0ee0aa3880b None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:57.773270 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcada5a7-a8d7-4630-82c5-e0ee0aa3880b None admin] PATH_INFO: `/v3/projects/6da47929012341928c673a2e0d1e97f0` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:57.776711 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcada5a7-a8d7-4630-82c5-e0ee0aa3880b None admin] RBAC: Authorizing `identity:get_project(project_id=6da47929012341928c673a2e0d1e97f0)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:57.779497 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcada5a7-a8d7-4630-82c5-e0ee0aa3880b None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:57.784297 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c639a4-1d0f-4a23-923a-c59c7ca5883b None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:57.785718 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 213/418] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Aug 4 21:02:57 2022] GET /identity/v3/projects/6da47929012341928c673a2e0d1e97f0 => generated 344 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:57.787960 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcada5a7-a8d7-4630-82c5-e0ee0aa3880b None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:57.844917 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:57.854040 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6ac3f3-0e11-4a24-8a41-277260dd98b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:57.856849 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ac3f3-0e11-4a24-8a41-277260dd98b3 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:57.873990 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ac3f3-0e11-4a24-8a41-277260dd98b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:57.873990 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ac3f3-0e11-4a24-8a41-277260dd98b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:57.996119 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcada5a7-a8d7-4630-82c5-e0ee0aa3880b None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:58.008816 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6ac3f3-0e11-4a24-8a41-277260dd98b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:58.010376 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6ac3f3-0e11-4a24-8a41-277260dd98b3 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:58.043528 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced2effe-bb51-4654-b82c-b9c6521a4f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:58.044161 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:58.044596 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced2effe-bb51-4654-b82c-b9c6521a4f33 None admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:58.044902 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced2effe-bb51-4654-b82c-b9c6521a4f33 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:58.045388 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced2effe-bb51-4654-b82c-b9c6521a4f33 None admin] PATH_INFO: `/v3/projects/6da47929012341928c673a2e0d1e97f0/groups/0117a08ce6764930bb139519a935554d/roles/a432dd4735944ecfab2fa5ce1705d356` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:58.047391 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced2effe-bb51-4654-b82c-b9c6521a4f33 None admin] RBAC: Authorizing `identity:create_grant(project_id=6da47929012341928c673a2e0d1e97f0, group_id=0117a08ce6764930bb139519a935554d, role_id=a432dd4735944ecfab2fa5ce1705d356)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:58.048853 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced2effe-bb51-4654-b82c-b9c6521a4f33 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:58.132331 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 214/419] 10.222.0.25 () {58 vars in 1442 bytes} [Thu Aug 4 21:02:57 2022] PUT /identity/v3/projects/6da47929012341928c673a2e0d1e97f0/groups/0117a08ce6764930bb139519a935554d/roles/a432dd4735944ecfab2fa5ce1705d356 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:02:58.153549 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 206/420] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Aug 4 21:02:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 556 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 21:02:58.159795 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced2effe-bb51-4654-b82c-b9c6521a4f33 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:58.312052 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:58.328221 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced2effe-bb51-4654-b82c-b9c6521a4f33 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:58.365752 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97a527a-ffc4-43b4-ac7a-8095b1a7343b None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:58.366094 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97a527a-ffc4-43b4-ac7a-8095b1a7343b None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:58.366321 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97a527a-ffc4-43b4-ac7a-8095b1a7343b None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:58.368851 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 207/421] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:02:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 21:02:58.373037 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e88bad-3911-43ee-9f00-eb6c65e0a953 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:58.373330 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e88bad-3911-43ee-9f00-eb6c65e0a953 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:58.373605 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e88bad-3911-43ee-9f00-eb6c65e0a953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:58.385186 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aad088-f9e5-4bd9-b126-dcda48e711b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:58.385857 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aad088-f9e5-4bd9-b126-dcda48e711b5 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:58.386143 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aad088-f9e5-4bd9-b126-dcda48e711b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:58.386418 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aad088-f9e5-4bd9-b126-dcda48e711b5 None admin] PATH_INFO: `/v3/projects/b8f804a236884f039371233cce01ecef` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:58.387666 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aad088-f9e5-4bd9-b126-dcda48e711b5 None admin] RBAC: Authorizing `identity:get_project(project_id=b8f804a236884f039371233cce01ecef)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:58.389277 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aad088-f9e5-4bd9-b126-dcda48e711b5 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:58.391166 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 215/422] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Aug 4 21:02:58 2022] GET /identity/v3/projects/b8f804a236884f039371233cce01ecef => generated 298 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:58.397143 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-81e88bad-3911-43ee-9f00-eb6c65e0a953 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:58.398358 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-81e88bad-3911-43ee-9f00-eb6c65e0a953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:58.403583 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aad088-f9e5-4bd9-b126-dcda48e711b5 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:58.497181 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 208/423] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:02:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 21:02:58.532134 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e88bad-3911-43ee-9f00-eb6c65e0a953 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:58.552773 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aad088-f9e5-4bd9-b126-dcda48e711b5 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:58.589130 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba95f3b-f268-4a25-8c55-61fd512eebd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:58.589701 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba95f3b-f268-4a25-8c55-61fd512eebd3 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:58.589993 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba95f3b-f268-4a25-8c55-61fd512eebd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:58.590267 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba95f3b-f268-4a25-8c55-61fd512eebd3 None admin] PATH_INFO: `/v3/groups/ba5e1c67c05e4d9582187eb89cd1acb5` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:58.591477 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba95f3b-f268-4a25-8c55-61fd512eebd3 None admin] RBAC: Authorizing `identity:get_group(group_id=ba5e1c67c05e4d9582187eb89cd1acb5)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:58.592923 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba95f3b-f268-4a25-8c55-61fd512eebd3 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:58.594970 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 216/424] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Aug 4 21:02:58 2022] GET /identity/v3/groups/ba5e1c67c05e4d9582187eb89cd1acb5 => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:02:58.608609 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba95f3b-f268-4a25-8c55-61fd512eebd3 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:58.615826 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:58.642685 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:58.728582 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e88bad-3911-43ee-9f00-eb6c65e0a953 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:58.785935 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba95f3b-f268-4a25-8c55-61fd512eebd3 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:58.795882 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c957675-29c1-402d-90de-0d55208f5368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:58.797038 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c957675-29c1-402d-90de-0d55208f5368 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:58.797461 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c957675-29c1-402d-90de-0d55208f5368 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:58.797873 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c957675-29c1-402d-90de-0d55208f5368 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:58.809361 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c957675-29c1-402d-90de-0d55208f5368 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:02:58.814602 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-3c957675-29c1-402d-90de-0d55208f5368 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 21:02:58.814602 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:02:58.814602 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:02:58.814602 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:02:58.814602 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:02:58.814602 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:02:58.814602 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:02:58.814602 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:02:58.814602 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:02:58.814602 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:02:58.814602 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:02:58.814602 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:02:58.814602 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:02:58.814602 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:02:58.814602 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:02:58.814602 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:02:58.816230 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:02:58.816230 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:02:58.816230 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:02:58.816230 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:02:58.816230 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:02:58.816230 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:02:58.816230 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:02:58.816230 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:02:58.816230 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:02:58.816230 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:02:58.816230 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:02:58.816230 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:02:58.816230 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:02:58.816230 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:02:58.816230 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:02:58.816230 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:02:58.817071 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:02:58.817071 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:02:58.817071 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:02:58.817071 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:02:58.817071 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:02:58.817071 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:02:58.817071 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:02:58.817071 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:02:58.817071 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:02:58.817071 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:02:58.817071 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 21:02:58.817071 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:02:58.819358 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 209/425] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Aug 4 21:02:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:02:58.832563 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c957675-29c1-402d-90de-0d55208f5368 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:58.851427 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ce48ff-5b22-4352-845c-6b3eed5ea8c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:58.851427 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ce48ff-5b22-4352-845c-6b3eed5ea8c6 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:58.851427 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ce48ff-5b22-4352-845c-6b3eed5ea8c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:58.851427 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ce48ff-5b22-4352-845c-6b3eed5ea8c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:58.851771 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ce48ff-5b22-4352-845c-6b3eed5ea8c6 None admin] RBAC: Adding query filter params (group.id=ba5e1c67c05e4d9582187eb89cd1acb5, role.id=8feebc60363c4feb870718b1721e5324, scope.project.id=b8f804a236884f039371233cce01ecef) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:58.851771 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ce48ff-5b22-4352-845c-6b3eed5ea8c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:58.854640 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ce48ff-5b22-4352-845c-6b3eed5ea8c6 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:58.861727 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 217/426] 10.222.0.25 () {56 vars in 1376 bytes} [Thu Aug 4 21:02:58 2022] GET /identity/v3/role_assignments?group.id=ba5e1c67c05e4d9582187eb89cd1acb5&scope.project.id=b8f804a236884f039371233cce01ecef&role.id=8feebc60363c4feb870718b1721e5324 => generated 611 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:02:58.926934 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f8a7f-915c-4715-990c-1f3b6f85b872 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:58.927260 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f8a7f-915c-4715-990c-1f3b6f85b872 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:58.928782 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f8a7f-915c-4715-990c-1f3b6f85b872 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:58.929324 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 218/427] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:02:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 21:02:58.934919 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e64455d-2f69-4f95-b5cc-fc3b211017a3 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:58.935213 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e64455d-2f69-4f95-b5cc-fc3b211017a3 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:58.935520 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e64455d-2f69-4f95-b5cc-fc3b211017a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:58.961453 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-7e64455d-2f69-4f95-b5cc-fc3b211017a3 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:58.963267 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-7e64455d-2f69-4f95-b5cc-fc3b211017a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:59.043196 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c957675-29c1-402d-90de-0d55208f5368 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:59.052600 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:59.075294 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 219/428] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:02:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 21:02:59.100186 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c422adee-088c-434b-92a2-4fbcc64bd199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:59.100779 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c422adee-088c-434b-92a2-4fbcc64bd199 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:59.101065 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c422adee-088c-434b-92a2-4fbcc64bd199 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:59.101223 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e64455d-2f69-4f95-b5cc-fc3b211017a3 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:59.101375 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c422adee-088c-434b-92a2-4fbcc64bd199 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:59.102113 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c422adee-088c-434b-92a2-4fbcc64bd199 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:59.102367 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c422adee-088c-434b-92a2-4fbcc64bd199 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:59.103803 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c422adee-088c-434b-92a2-4fbcc64bd199 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:59.116235 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 210/429] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Aug 4 21:02:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:59.121728 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c422adee-088c-434b-92a2-4fbcc64bd199 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:59.122889 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:59.233771 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e64455d-2f69-4f95-b5cc-fc3b211017a3 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:59.246857 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c422adee-088c-434b-92a2-4fbcc64bd199 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:59.278351 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ea5142-fdce-4a21-96ae-ceccd75ab8a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:59.279002 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ea5142-fdce-4a21-96ae-ceccd75ab8a7 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:59.279282 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ea5142-fdce-4a21-96ae-ceccd75ab8a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:59.279591 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ea5142-fdce-4a21-96ae-ceccd75ab8a7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:59.289298 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abbbef4-133b-4270-8a62-e9c51aa5c978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:59.290206 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abbbef4-133b-4270-8a62-e9c51aa5c978 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:59.290586 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abbbef4-133b-4270-8a62-e9c51aa5c978 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:59.290913 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abbbef4-133b-4270-8a62-e9c51aa5c978 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:59.292176 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ea5142-fdce-4a21-96ae-ceccd75ab8a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:02:59.298216 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-33ea5142-fdce-4a21-96ae-ceccd75ab8a7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 21:02:59.298216 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:02:59.298216 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:02:59.298216 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:02:59.298216 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:02:59.298216 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:02:59.298216 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:02:59.298216 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:02:59.298216 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:02:59.298216 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:02:59.298216 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:02:59.298216 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:02:59.298216 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:02:59.298216 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:02:59.298216 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:02:59.298216 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:02:59.299146 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:02:59.299146 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:02:59.299146 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:02:59.299146 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:02:59.299146 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:02:59.299146 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:02:59.299146 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:02:59.299146 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:02:59.299146 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:02:59.299146 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:02:59.299146 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:02:59.299146 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:02:59.299146 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:02:59.299146 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:02:59.299146 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:02:59.299146 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:02:59.300547 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:02:59.300547 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:02:59.300547 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:02:59.300547 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:02:59.300547 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:02:59.300547 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:02:59.300547 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:02:59.300547 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:02:59.300547 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:02:59.300547 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:02:59.300547 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 21:02:59.300547 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:02:59.300547 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 220/430] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Aug 4 21:02:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:02:59.301686 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa865c0d-aecb-4a38-837d-5c47ebc0f89e None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:59.301974 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa865c0d-aecb-4a38-837d-5c47ebc0f89e None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:59.302252 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa865c0d-aecb-4a38-837d-5c47ebc0f89e None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:59.307562 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 221/431] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:02:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:02:59.308452 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa865c0d-aecb-4a38-837d-5c47ebc0f89e None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:59.345393 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:59.391685 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abbbef4-133b-4270-8a62-e9c51aa5c978 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:59.393139 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abbbef4-133b-4270-8a62-e9c51aa5c978 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:59.408390 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:59.440312 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa865c0d-aecb-4a38-837d-5c47ebc0f89e None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:59.483513 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7663371-a4ca-4275-a25b-cbfbe4b04d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:59.484466 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7663371-a4ca-4275-a25b-cbfbe4b04d4f None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:59.484923 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7663371-a4ca-4275-a25b-cbfbe4b04d4f None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:59.485367 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7663371-a4ca-4275-a25b-cbfbe4b04d4f None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:59.486727 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7663371-a4ca-4275-a25b-cbfbe4b04d4f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:59.486727 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7663371-a4ca-4275-a25b-cbfbe4b04d4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:59.488338 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7663371-a4ca-4275-a25b-cbfbe4b04d4f None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:59.497695 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 222/432] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Aug 4 21:02:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:59.498784 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08b3207-4198-45cf-93c9-87bcc65f3894 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:59.499071 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08b3207-4198-45cf-93c9-87bcc65f3894 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:59.499395 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08b3207-4198-45cf-93c9-87bcc65f3894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:59.517826 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-e08b3207-4198-45cf-93c9-87bcc65f3894 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:59.519034 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-e08b3207-4198-45cf-93c9-87bcc65f3894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:59.532311 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 211/433] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Aug 4 21:02:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 04 21:02:59.533228 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abbbef4-133b-4270-8a62-e9c51aa5c978 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:59.594067 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 223/434] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:02:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:02:59.594905 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08b3207-4198-45cf-93c9-87bcc65f3894 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:59.615298 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:59.652687 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abbbef4-133b-4270-8a62-e9c51aa5c978 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:59.679421 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:59.689097 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a7da90-cafe-4af3-a0a3-836460187c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:59.689913 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a7da90-cafe-4af3-a0a3-836460187c98 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:59.691918 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a7da90-cafe-4af3-a0a3-836460187c98 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:59.691918 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a7da90-cafe-4af3-a0a3-836460187c98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:59.712431 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08b3207-4198-45cf-93c9-87bcc65f3894 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:59.747382 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590fbcb8-1c52-4d44-9288-db4659dab426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:59.748027 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590fbcb8-1c52-4d44-9288-db4659dab426 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:59.748313 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590fbcb8-1c52-4d44-9288-db4659dab426 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:59.748592 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590fbcb8-1c52-4d44-9288-db4659dab426 None admin] PATH_INFO: `/v3/projects/b8f804a236884f039371233cce01ecef` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:59.749752 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590fbcb8-1c52-4d44-9288-db4659dab426 None admin] RBAC: Authorizing `identity:get_project(project_id=b8f804a236884f039371233cce01ecef)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:59.751193 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590fbcb8-1c52-4d44-9288-db4659dab426 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:59.753491 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 224/435] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Aug 4 21:02:59 2022] GET /identity/v3/projects/b8f804a236884f039371233cce01ecef => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:59.754243 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590fbcb8-1c52-4d44-9288-db4659dab426 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:59.765862 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a7da90-cafe-4af3-a0a3-836460187c98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:59.767466 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a7da90-cafe-4af3-a0a3-836460187c98 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:59.838803 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:59.856517 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590fbcb8-1c52-4d44-9288-db4659dab426 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:59.873922 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 212/436] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Aug 4 21:02:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 21:02:59.875165 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a7da90-cafe-4af3-a0a3-836460187c98 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:59.897261 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958199f8-c599-48b9-8779-57d1ddf27750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:59.897911 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958199f8-c599-48b9-8779-57d1ddf27750 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:59.898195 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958199f8-c599-48b9-8779-57d1ddf27750 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:59.898469 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958199f8-c599-48b9-8779-57d1ddf27750 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:59.906652 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:59.911639 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958199f8-c599-48b9-8779-57d1ddf27750 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:02:59.916620 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-958199f8-c599-48b9-8779-57d1ddf27750 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 21:02:59.916620 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:02:59.916620 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:02:59.916620 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:02:59.916620 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:02:59.916620 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:02:59.916620 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:02:59.916620 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:02:59.916620 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:02:59.916620 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:02:59.916620 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:02:59.916620 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:02:59.916620 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:02:59.916620 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:02:59.916620 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:02:59.916620 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:02:59.918131 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:02:59.918131 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:02:59.918131 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:02:59.918131 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:02:59.918131 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:02:59.918131 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:02:59.918131 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:02:59.918131 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:02:59.918131 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:02:59.918131 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:02:59.918131 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:02:59.918131 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:02:59.918131 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:02:59.918131 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:02:59.918131 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:02:59.918131 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:02:59.918911 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:02:59.918911 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:02:59.918911 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:02:59.918911 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:02:59.918911 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:02:59.918911 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:02:59.918911 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:02:59.918911 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:02:59.918911 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:02:59.918911 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:02:59.918911 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 21:02:59.918911 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:02:59.918911 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 225/437] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Aug 4 21:02:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:02:59.918911 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958199f8-c599-48b9-8779-57d1ddf27750 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:59.982461 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a7da90-cafe-4af3-a0a3-836460187c98 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:00.018938 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf46156-4087-4756-8017-96b2c6dd9736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:00.019617 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf46156-4087-4756-8017-96b2c6dd9736 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:00.019903 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf46156-4087-4756-8017-96b2c6dd9736 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:00.020177 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf46156-4087-4756-8017-96b2c6dd9736 None admin] PATH_INFO: `/v3/groups/ba5e1c67c05e4d9582187eb89cd1acb5` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:00.021395 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf46156-4087-4756-8017-96b2c6dd9736 None admin] RBAC: Authorizing `identity:get_group(group_id=ba5e1c67c05e4d9582187eb89cd1acb5)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:00.022965 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf46156-4087-4756-8017-96b2c6dd9736 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:00.029444 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 213/438] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Aug 4 21:02:59 2022] GET /identity/v3/groups/ba5e1c67c05e4d9582187eb89cd1acb5 => generated 226 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:00.029444 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf46156-4087-4756-8017-96b2c6dd9736 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:00.051221 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958199f8-c599-48b9-8779-57d1ddf27750 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:00.063036 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:00.085028 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1218ad4d-75bb-4dda-8655-df48c0a7ea16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:00.085677 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1218ad4d-75bb-4dda-8655-df48c0a7ea16 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:00.085956 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1218ad4d-75bb-4dda-8655-df48c0a7ea16 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:00.086223 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1218ad4d-75bb-4dda-8655-df48c0a7ea16 None admin] PATH_INFO: `/v3/projects/6c2de9abe1504f6698a9ea2a032c91e4` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:00.087401 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1218ad4d-75bb-4dda-8655-df48c0a7ea16 None admin] RBAC: Authorizing `identity:get_project(project_id=6c2de9abe1504f6698a9ea2a032c91e4)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:00.088839 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1218ad4d-75bb-4dda-8655-df48c0a7ea16 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:00.094894 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 226/439] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Aug 4 21:02:59 2022] GET /identity/v3/projects/6c2de9abe1504f6698a9ea2a032c91e4 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:00.094894 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1218ad4d-75bb-4dda-8655-df48c0a7ea16 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:00.138647 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf46156-4087-4756-8017-96b2c6dd9736 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:00.157729 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:00.177967 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a211ebe-7e47-4daa-bf12-e1265c9c07b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:00.178637 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a211ebe-7e47-4daa-bf12-e1265c9c07b8 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:00.178912 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a211ebe-7e47-4daa-bf12-e1265c9c07b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:00.179184 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a211ebe-7e47-4daa-bf12-e1265c9c07b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:00.179922 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a211ebe-7e47-4daa-bf12-e1265c9c07b8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:00.180230 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a211ebe-7e47-4daa-bf12-e1265c9c07b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:00.181610 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a211ebe-7e47-4daa-bf12-e1265c9c07b8 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:00.194989 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 214/440] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Aug 4 21:03:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:00.194989 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a211ebe-7e47-4daa-bf12-e1265c9c07b8 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:00.202125 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1218ad4d-75bb-4dda-8655-df48c0a7ea16 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:00.240001 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da25ef2-990c-47bd-8bf1-2f97fbc73406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:00.240699 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da25ef2-990c-47bd-8bf1-2f97fbc73406 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:00.240980 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da25ef2-990c-47bd-8bf1-2f97fbc73406 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:00.241251 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da25ef2-990c-47bd-8bf1-2f97fbc73406 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:00.242160 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da25ef2-990c-47bd-8bf1-2f97fbc73406 None admin] RBAC: Adding query filter params (group.id=ba5e1c67c05e4d9582187eb89cd1acb5, role.id=70587326d8584316b73b380df2ae8e67, scope.project.id=b8f804a236884f039371233cce01ecef) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:00.242469 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da25ef2-990c-47bd-8bf1-2f97fbc73406 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:00.243931 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da25ef2-990c-47bd-8bf1-2f97fbc73406 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:00.250538 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 227/441] 10.222.0.25 () {56 vars in 1376 bytes} [Thu Aug 4 21:03:00 2022] GET /identity/v3/role_assignments?group.id=ba5e1c67c05e4d9582187eb89cd1acb5&scope.project.id=b8f804a236884f039371233cce01ecef&role.id=70587326d8584316b73b380df2ae8e67 => generated 611 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:00.251141 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da25ef2-990c-47bd-8bf1-2f97fbc73406 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:00.320413 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:00.329031 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a211ebe-7e47-4daa-bf12-e1265c9c07b8 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:00.375029 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da25ef2-990c-47bd-8bf1-2f97fbc73406 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:00.379303 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b37aa7-618e-4f6c-a736-0c484362a9eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:00.380287 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b37aa7-618e-4f6c-a736-0c484362a9eb None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:00.380753 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b37aa7-618e-4f6c-a736-0c484362a9eb None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:00.381217 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b37aa7-618e-4f6c-a736-0c484362a9eb None admin] PATH_INFO: `/v3/groups/ba5e1c67c05e4d9582187eb89cd1acb5` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:00.382718 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b37aa7-618e-4f6c-a736-0c484362a9eb None admin] RBAC: Authorizing `identity:get_group(group_id=ba5e1c67c05e4d9582187eb89cd1acb5)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:00.384811 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b37aa7-618e-4f6c-a736-0c484362a9eb None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:00.391375 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 215/442] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Aug 4 21:03:00 2022] GET /identity/v3/groups/ba5e1c67c05e4d9582187eb89cd1acb5 => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:00.392378 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b37aa7-618e-4f6c-a736-0c484362a9eb None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:00.418856 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:00.426299 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb57005-7448-4649-8af9-d90838f6d33f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:00.426709 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb57005-7448-4649-8af9-d90838f6d33f None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:00.430282 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb57005-7448-4649-8af9-d90838f6d33f None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:00.430282 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb57005-7448-4649-8af9-d90838f6d33f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:00.515589 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb57005-7448-4649-8af9-d90838f6d33f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:00.517048 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb57005-7448-4649-8af9-d90838f6d33f None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:00.528489 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b37aa7-618e-4f6c-a736-0c484362a9eb None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:00.567180 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c28715-c0ce-4ea5-8ebc-f517cd41155c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:00.567826 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c28715-c0ce-4ea5-8ebc-f517cd41155c None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:00.568109 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c28715-c0ce-4ea5-8ebc-f517cd41155c None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:00.568408 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c28715-c0ce-4ea5-8ebc-f517cd41155c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:00.569237 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c28715-c0ce-4ea5-8ebc-f517cd41155c None admin] RBAC: Adding query filter params (group.id=ba5e1c67c05e4d9582187eb89cd1acb5, role.id=70587326d8584316b73b380df2ae8e67, scope.project.id=6c2de9abe1504f6698a9ea2a032c91e4) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:00.569551 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c28715-c0ce-4ea5-8ebc-f517cd41155c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:00.570999 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c28715-c0ce-4ea5-8ebc-f517cd41155c None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:00.577123 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 216/443] 10.222.0.25 () {56 vars in 1376 bytes} [Thu Aug 4 21:03:00 2022] GET /identity/v3/role_assignments?group.id=ba5e1c67c05e4d9582187eb89cd1acb5&scope.project.id=6c2de9abe1504f6698a9ea2a032c91e4&role.id=70587326d8584316b73b380df2ae8e67 => generated 611 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:00.578230 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816e0ad3-a428-4b70-8837-e7cea32e8228 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:00.578569 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816e0ad3-a428-4b70-8837-e7cea32e8228 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:00.578848 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816e0ad3-a428-4b70-8837-e7cea32e8228 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:00.580434 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 217/444] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:03:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:03:00.586681 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc029ee-e124-46b3-b271-17998ad91209 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:00.586992 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc029ee-e124-46b3-b271-17998ad91209 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:00.587248 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc029ee-e124-46b3-b271-17998ad91209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:00.614761 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-5bc029ee-e124-46b3-b271-17998ad91209 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:03:00.616028 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-5bc029ee-e124-46b3-b271-17998ad91209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:03:00.645346 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:00.664071 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 228/445] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Aug 4 21:03:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 21:03:00.669973 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb57005-7448-4649-8af9-d90838f6d33f None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:00.702610 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:00.711401 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 218/446] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:03:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:03:00.729776 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc029ee-e124-46b3-b271-17998ad91209 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:00.789733 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb57005-7448-4649-8af9-d90838f6d33f None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:00.831770 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc65f4be-9939-4d16-ae5b-39d6e0967e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:00.832580 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc65f4be-9939-4d16-ae5b-39d6e0967e52 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:00.832869 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc65f4be-9939-4d16-ae5b-39d6e0967e52 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:00.833180 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc65f4be-9939-4d16-ae5b-39d6e0967e52 None admin] PATH_INFO: `/v3/projects/6c2de9abe1504f6698a9ea2a032c91e4` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:00.834351 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc65f4be-9939-4d16-ae5b-39d6e0967e52 None admin] RBAC: Authorizing `identity:get_project(project_id=6c2de9abe1504f6698a9ea2a032c91e4)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:00.835857 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc65f4be-9939-4d16-ae5b-39d6e0967e52 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:00.837556 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 229/447] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Aug 4 21:03:00 2022] GET /identity/v3/projects/6c2de9abe1504f6698a9ea2a032c91e4 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:03:00.838327 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc029ee-e124-46b3-b271-17998ad91209 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:00.842941 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc65f4be-9939-4d16-ae5b-39d6e0967e52 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:00.874667 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b96123-088c-4866-89e6-e0548b397562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:00.875331 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b96123-088c-4866-89e6-e0548b397562 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:00.875614 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b96123-088c-4866-89e6-e0548b397562 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:00.875893 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b96123-088c-4866-89e6-e0548b397562 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:00.887535 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b96123-088c-4866-89e6-e0548b397562 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:03:00.892440 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-52b96123-088c-4866-89e6-e0548b397562 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:03:00.892440 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:00.892440 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:00.892440 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:00.892440 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:00.892440 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:00.892440 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:00.892440 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:00.892440 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:00.892440 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:00.892440 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:00.892440 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:00.892440 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:03:00.892440 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:03:00.892440 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:03:00.892440 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:03:00.894063 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:03:00.894063 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:03:00.894063 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:03:00.894063 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:03:00.894063 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:00.894063 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:00.894063 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:00.894063 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:00.894063 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:00.894063 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:00.894063 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:00.894063 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:00.894063 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:00.894063 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:00.894063 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:00.894063 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:00.895044 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:00.895044 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:00.895044 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:00.895044 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:00.895044 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:03:00.895044 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:03:00.895044 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:03:00.895044 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:03:00.895044 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:03:00.895044 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:03:00.895044 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:03:00.895044 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:00.896138 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 219/448] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Aug 4 21:03:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:03:00.900290 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b96123-088c-4866-89e6-e0548b397562 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:00.919378 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:00.944530 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:00.948400 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc65f4be-9939-4d16-ae5b-39d6e0967e52 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:00.983915 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca595c0-2982-4945-9098-976805888b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:00.984551 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca595c0-2982-4945-9098-976805888b18 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:00.984828 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca595c0-2982-4945-9098-976805888b18 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:00.985099 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca595c0-2982-4945-9098-976805888b18 None admin] PATH_INFO: `/v3/groups/ba5e1c67c05e4d9582187eb89cd1acb5` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:00.986235 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca595c0-2982-4945-9098-976805888b18 None admin] RBAC: Authorizing `identity:get_group(group_id=ba5e1c67c05e4d9582187eb89cd1acb5)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:00.988052 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca595c0-2982-4945-9098-976805888b18 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:00.991888 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 230/449] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Aug 4 21:03:00 2022] GET /identity/v3/groups/ba5e1c67c05e4d9582187eb89cd1acb5 => generated 226 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:03:00.995651 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca595c0-2982-4945-9098-976805888b18 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:00.998697 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b96123-088c-4866-89e6-e0548b397562 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:01.034840 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578a59b6-46a0-4994-ab1c-f2158afa7cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:01.035491 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a59b6-46a0-4994-ab1c-f2158afa7cb9 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:01.035775 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a59b6-46a0-4994-ab1c-f2158afa7cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:01.036048 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a59b6-46a0-4994-ab1c-f2158afa7cb9 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:01.036763 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578a59b6-46a0-4994-ab1c-f2158afa7cb9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:01.037068 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578a59b6-46a0-4994-ab1c-f2158afa7cb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:01.038482 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578a59b6-46a0-4994-ab1c-f2158afa7cb9 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:01.049864 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 220/450] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Aug 4 21:03:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:03:01.055487 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578a59b6-46a0-4994-ab1c-f2158afa7cb9 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:01.097448 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca595c0-2982-4945-9098-976805888b18 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:01.131112 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4367a6ef-1884-4404-b20c-73b29dfcc255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:01.131764 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4367a6ef-1884-4404-b20c-73b29dfcc255 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:01.132047 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4367a6ef-1884-4404-b20c-73b29dfcc255 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:01.132324 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4367a6ef-1884-4404-b20c-73b29dfcc255 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:01.133110 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4367a6ef-1884-4404-b20c-73b29dfcc255 None admin] RBAC: Adding query filter params (group.id=ba5e1c67c05e4d9582187eb89cd1acb5, role.id=8feebc60363c4feb870718b1721e5324, scope.project.id=6c2de9abe1504f6698a9ea2a032c91e4) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:01.133419 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4367a6ef-1884-4404-b20c-73b29dfcc255 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:01.134850 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4367a6ef-1884-4404-b20c-73b29dfcc255 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:01.140734 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 231/451] 10.222.0.25 () {56 vars in 1376 bytes} [Thu Aug 4 21:03:00 2022] GET /identity/v3/role_assignments?group.id=ba5e1c67c05e4d9582187eb89cd1acb5&scope.project.id=6c2de9abe1504f6698a9ea2a032c91e4&role.id=8feebc60363c4feb870718b1721e5324 => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:03:01.143058 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:01.162642 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578a59b6-46a0-4994-ab1c-f2158afa7cb9 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:01.201452 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd0a8eb-713f-4b39-8b9b-2964f9dcab57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:01.202007 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0a8eb-713f-4b39-8b9b-2964f9dcab57 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:01.202287 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0a8eb-713f-4b39-8b9b-2964f9dcab57 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:01.202560 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0a8eb-713f-4b39-8b9b-2964f9dcab57 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:01.277055 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd0a8eb-713f-4b39-8b9b-2964f9dcab57 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:01.278534 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd0a8eb-713f-4b39-8b9b-2964f9dcab57 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:01.359105 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:01.381105 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 221/452] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Aug 4 21:03:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 04 21:03:01.386759 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4367a6ef-1884-4404-b20c-73b29dfcc255 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:01.418920 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:01.491231 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4367a6ef-1884-4404-b20c-73b29dfcc255 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:01.525494 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54e7e9e-7fed-4f75-8e26-e05335b74e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:01.526138 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54e7e9e-7fed-4f75-8e26-e05335b74e3d None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:01.526420 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54e7e9e-7fed-4f75-8e26-e05335b74e3d None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:01.526694 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54e7e9e-7fed-4f75-8e26-e05335b74e3d None admin] PATH_INFO: `/v3/projects/6da47929012341928c673a2e0d1e97f0` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:01.527930 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54e7e9e-7fed-4f75-8e26-e05335b74e3d None admin] RBAC: Authorizing `identity:get_project(project_id=6da47929012341928c673a2e0d1e97f0)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:01.529362 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54e7e9e-7fed-4f75-8e26-e05335b74e3d None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:01.531185 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 232/453] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Aug 4 21:03:01 2022] GET /identity/v3/projects/6da47929012341928c673a2e0d1e97f0 => generated 344 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:01.537567 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd0a8eb-713f-4b39-8b9b-2964f9dcab57 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:01.637195 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd0a8eb-713f-4b39-8b9b-2964f9dcab57 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:01.670009 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874d9b9f-b51b-4f0d-8ee4-fccd249ed79f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:01.670672 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874d9b9f-b51b-4f0d-8ee4-fccd249ed79f None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:01.670956 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874d9b9f-b51b-4f0d-8ee4-fccd249ed79f None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:01.671232 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874d9b9f-b51b-4f0d-8ee4-fccd249ed79f None admin] PATH_INFO: `/v3/groups/0117a08ce6764930bb139519a935554d` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:01.672399 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874d9b9f-b51b-4f0d-8ee4-fccd249ed79f None admin] RBAC: Authorizing `identity:get_group(group_id=0117a08ce6764930bb139519a935554d)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:01.673835 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874d9b9f-b51b-4f0d-8ee4-fccd249ed79f None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:01.676077 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 222/454] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Aug 4 21:03:01 2022] GET /identity/v3/groups/0117a08ce6764930bb139519a935554d => generated 229 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:01.681864 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54e7e9e-7fed-4f75-8e26-e05335b74e3d None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:01.775004 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:01.778610 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54e7e9e-7fed-4f75-8e26-e05335b74e3d None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:01.811919 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffdc7f9-632d-4dba-ab3a-e0b7beae946f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:01.812531 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffdc7f9-632d-4dba-ab3a-e0b7beae946f None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:01.812817 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffdc7f9-632d-4dba-ab3a-e0b7beae946f None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:01.813094 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffdc7f9-632d-4dba-ab3a-e0b7beae946f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:01.813879 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffdc7f9-632d-4dba-ab3a-e0b7beae946f None admin] RBAC: Adding query filter params (group.id=0117a08ce6764930bb139519a935554d, role.id=a432dd4735944ecfab2fa5ce1705d356, scope.project.id=6da47929012341928c673a2e0d1e97f0) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:01.814193 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffdc7f9-632d-4dba-ab3a-e0b7beae946f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:01.815674 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffdc7f9-632d-4dba-ab3a-e0b7beae946f None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:01.821255 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 233/455] 10.222.0.25 () {56 vars in 1376 bytes} [Thu Aug 4 21:03:01 2022] GET /identity/v3/role_assignments?group.id=0117a08ce6764930bb139519a935554d&scope.project.id=6da47929012341928c673a2e0d1e97f0&role.id=a432dd4735944ecfab2fa5ce1705d356 => generated 611 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:04.454291 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7a3c89-5bc2-42c2-8c28-dd191c09c9b6 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:04.454791 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7a3c89-5bc2-42c2-8c28-dd191c09c9b6 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:04.454865 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7a3c89-5bc2-42c2-8c28-dd191c09c9b6 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:04.456741 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 223/456] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:03:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:03:04.461890 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d76d2b-f3c3-4d07-b5f6-4287f1f0abf4 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:04.462316 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d76d2b-f3c3-4d07-b5f6-4287f1f0abf4 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:04.462481 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d76d2b-f3c3-4d07-b5f6-4287f1f0abf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:04.483776 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-e5d76d2b-f3c3-4d07-b5f6-4287f1f0abf4 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:03:04.484864 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-e5d76d2b-f3c3-4d07-b5f6-4287f1f0abf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:03:04.489401 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d70618-873b-4e63-8004-a3b42febc729 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:04.490771 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d70618-873b-4e63-8004-a3b42febc729 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:04.491258 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d70618-873b-4e63-8004-a3b42febc729 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:04.493090 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 224/457] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:03:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:03:04.506463 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c5cd59-b963-4e6c-8526-7d27e53253ba None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:04.507030 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c5cd59-b963-4e6c-8526-7d27e53253ba None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:04.507566 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c5cd59-b963-4e6c-8526-7d27e53253ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:04.538441 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-39c5cd59-b963-4e6c-8526-7d27e53253ba None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:03:04.540622 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-39c5cd59-b963-4e6c-8526-7d27e53253ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:03:04.593246 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 234/458] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:03:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:03:04.595036 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21e5983-42bf-4f1d-9e74-163fd502e81d None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:04.595549 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21e5983-42bf-4f1d-9e74-163fd502e81d None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:04.596591 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21e5983-42bf-4f1d-9e74-163fd502e81d None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:04.598293 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 235/459] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:03:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:03:04.605012 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686eca17-f4cc-4cff-b60e-7e2b8deb3e9d None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:04.605012 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686eca17-f4cc-4cff-b60e-7e2b8deb3e9d None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:04.605012 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686eca17-f4cc-4cff-b60e-7e2b8deb3e9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:04.613437 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:04.632236 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-686eca17-f4cc-4cff-b60e-7e2b8deb3e9d None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:03:04.633737 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-686eca17-f4cc-4cff-b60e-7e2b8deb3e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:03:04.644974 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 225/460] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:03:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 21:03:04.645828 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c5cd59-b963-4e6c-8526-7d27e53253ba None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:04.729961 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 236/461] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:03:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 21:03:04.730846 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686eca17-f4cc-4cff-b60e-7e2b8deb3e9d None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:04.770989 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:04.782775 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c5cd59-b963-4e6c-8526-7d27e53253ba None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:04.829846 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3145492-5ad6-45b0-9f02-9f5de97ae2ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:04.830546 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3145492-5ad6-45b0-9f02-9f5de97ae2ec None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:04.831092 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3145492-5ad6-45b0-9f02-9f5de97ae2ec None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:04.831402 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3145492-5ad6-45b0-9f02-9f5de97ae2ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:04.847730 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3145492-5ad6-45b0-9f02-9f5de97ae2ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:04.849883 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3145492-5ad6-45b0-9f02-9f5de97ae2ec None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:04.855211 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686eca17-f4cc-4cff-b60e-7e2b8deb3e9d None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:04.871485 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-c3145492-5ad6-45b0-9f02-9f5de97ae2ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:04.871485 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:04.871485 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:04.871485 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:04.871485 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:04.871485 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:04.871485 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:04.871485 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:04.871485 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:04.871485 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:04.871485 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:04.871485 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:04.871485 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:03:04.871485 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:03:04.871485 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:03:04.872549 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:03:04.872549 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:04.872549 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:04.872549 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:04.872549 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:04.872549 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:04.872549 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:04.872549 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:04.872549 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:04.872549 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:04.872549 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:04.872549 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:04.872549 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:04.872549 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:04.872549 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:04.872549 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:04.872549 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:04.873604 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:03:04.873604 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:03:04.873604 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:04.873604 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:04.875416 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 226/462] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:03:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:04.876255 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3145492-5ad6-45b0-9f02-9f5de97ae2ec None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:04.902768 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08197ed-bf26-4903-bcaf-6134d75035eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:04.903717 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08197ed-bf26-4903-bcaf-6134d75035eb None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:04.904002 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08197ed-bf26-4903-bcaf-6134d75035eb None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:04.904277 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08197ed-bf26-4903-bcaf-6134d75035eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:04.907491 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:04.925114 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08197ed-bf26-4903-bcaf-6134d75035eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:04.927339 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08197ed-bf26-4903-bcaf-6134d75035eb None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:04.946549 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-c08197ed-bf26-4903-bcaf-6134d75035eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:04.946549 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:04.946549 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:04.946549 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:04.946549 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:04.946549 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:04.946549 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:04.946549 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:04.946549 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:04.946549 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:04.946549 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:04.946549 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:04.946549 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:03:04.946549 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:03:04.946549 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:03:04.948591 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:03:04.948591 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:04.948591 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:04.948591 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:04.948591 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:04.948591 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:04.948591 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:04.948591 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:04.948591 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:04.948591 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:04.948591 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:04.948591 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:04.948591 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:04.948591 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:04.948591 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:04.948591 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:04.948591 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:04.950624 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:03:04.950624 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:03:04.950624 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:04.950624 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:04.956410 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 237/463] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:03:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:04.956410 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08197ed-bf26-4903-bcaf-6134d75035eb None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:04.986405 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3145492-5ad6-45b0-9f02-9f5de97ae2ec None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:05.023136 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2896effb-aba0-4479-9f58-e01f24a93906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:05.023842 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2896effb-aba0-4479-9f58-e01f24a93906 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:05.024129 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2896effb-aba0-4479-9f58-e01f24a93906 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:05.024403 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2896effb-aba0-4479-9f58-e01f24a93906 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:05.034445 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:05.037141 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2896effb-aba0-4479-9f58-e01f24a93906 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:05.038691 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2896effb-aba0-4479-9f58-e01f24a93906 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:05.055091 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-2896effb-aba0-4479-9f58-e01f24a93906 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:05.055091 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:05.055091 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:05.055091 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:05.055091 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:05.055091 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:05.055091 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:05.055091 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:05.055091 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:05.055091 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:05.055091 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:05.055091 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:05.055091 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:03:05.055091 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:03:05.055091 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:03:05.056971 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:03:05.056971 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:05.056971 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:05.056971 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:05.056971 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:05.056971 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:05.056971 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:05.056971 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:05.056971 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:05.056971 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:05.056971 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:05.056971 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:05.056971 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:05.056971 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:05.056971 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:05.056971 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:05.056971 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:05.059040 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:03:05.059040 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:03:05.059040 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:05.059040 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:05.059040 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 227/464] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:03:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:05.063388 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08197ed-bf26-4903-bcaf-6134d75035eb None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:05.063592 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2896effb-aba0-4479-9f58-e01f24a93906 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:05.098055 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef4b0f9-162a-43e7-a2d6-92a9dade46f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:05.098811 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef4b0f9-162a-43e7-a2d6-92a9dade46f7 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:05.099104 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef4b0f9-162a-43e7-a2d6-92a9dade46f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:05.099433 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef4b0f9-162a-43e7-a2d6-92a9dade46f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:05.100145 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef4b0f9-162a-43e7-a2d6-92a9dade46f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:05.100603 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef4b0f9-162a-43e7-a2d6-92a9dade46f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:05.102800 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef4b0f9-162a-43e7-a2d6-92a9dade46f7 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:05.123349 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 238/465] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:03:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:05.124272 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef4b0f9-162a-43e7-a2d6-92a9dade46f7 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:05.155241 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:05.175811 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2896effb-aba0-4479-9f58-e01f24a93906 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:05.212847 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040c8325-31ec-4451-925a-eb0cb854d227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:05.213756 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c8325-31ec-4451-925a-eb0cb854d227 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:05.214041 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c8325-31ec-4451-925a-eb0cb854d227 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:05.214330 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c8325-31ec-4451-925a-eb0cb854d227 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:05.215013 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040c8325-31ec-4451-925a-eb0cb854d227 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:05.215333 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040c8325-31ec-4451-925a-eb0cb854d227 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:05.216738 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040c8325-31ec-4451-925a-eb0cb854d227 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:05.235196 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 228/466] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:03:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:05.236058 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040c8325-31ec-4451-925a-eb0cb854d227 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:05.250832 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef4b0f9-162a-43e7-a2d6-92a9dade46f7 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:05.288167 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f7fdb8-801a-44f7-819a-b5514706f558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:05.288715 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f7fdb8-801a-44f7-819a-b5514706f558 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:05.289002 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f7fdb8-801a-44f7-819a-b5514706f558 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:05.289278 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f7fdb8-801a-44f7-819a-b5514706f558 None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:05.289973 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f7fdb8-801a-44f7-819a-b5514706f558 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:05.290275 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f7fdb8-801a-44f7-819a-b5514706f558 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:05.291674 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f7fdb8-801a-44f7-819a-b5514706f558 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:05.293235 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:05.307881 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 239/467] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:03:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:05.307881 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17109633-5beb-45ca-a85e-959cf192363b None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:05.307881 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17109633-5beb-45ca-a85e-959cf192363b None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:05.307881 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17109633-5beb-45ca-a85e-959cf192363b None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:05.307881 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 240/468] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:03:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:03:05.307881 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17109633-5beb-45ca-a85e-959cf192363b None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:05.351738 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040c8325-31ec-4451-925a-eb0cb854d227 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:05.387295 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd58199-fd93-4d31-94be-a0dbaf448cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:05.387938 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd58199-fd93-4d31-94be-a0dbaf448cb5 None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:05.388252 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd58199-fd93-4d31-94be-a0dbaf448cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:05.388501 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd58199-fd93-4d31-94be-a0dbaf448cb5 None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:05.389172 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd58199-fd93-4d31-94be-a0dbaf448cb5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:05.390629 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd58199-fd93-4d31-94be-a0dbaf448cb5 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:05.417796 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17109633-5beb-45ca-a85e-959cf192363b None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:05.451863 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ea1816-9468-4d07-b521-c245234ebfa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:05.452492 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ea1816-9468-4d07-b521-c245234ebfa0 None admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:05.452811 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ea1816-9468-4d07-b521-c245234ebfa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:05.453089 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ea1816-9468-4d07-b521-c245234ebfa0 None admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:05.453775 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ea1816-9468-4d07-b521-c245234ebfa0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:05.455284 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ea1816-9468-4d07-b521-c245234ebfa0 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:05.491142 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:05.496486 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 229/469] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Aug 4 21:03:05 2022] POST /identity/v3/users => generated 248 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:03:05.497710 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d6348-a812-4657-b121-fa10d7b3a005 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:05.499550 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d6348-a812-4657-b121-fa10d7b3a005 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:05.499550 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d6348-a812-4657-b121-fa10d7b3a005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:05.518639 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-4a8d6348-a812-4657-b121-fa10d7b3a005 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:03:05.519941 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-4a8d6348-a812-4657-b121-fa10d7b3a005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:03:05.549522 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 241/470] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Aug 4 21:03:05 2022] POST /identity/v3/users => generated 246 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:03:05.550934 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ea1816-9468-4d07-b521-c245234ebfa0 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:05.613076 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 230/471] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:03:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:03:05.614840 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72934aa3-51be-45fa-b861-be62ce3a8172 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:05.615237 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72934aa3-51be-45fa-b861-be62ce3a8172 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:05.615686 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72934aa3-51be-45fa-b861-be62ce3a8172 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:05.617560 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 231/472] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:03:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:03:05.622773 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a97cde-fed2-46d6-8105-982f4e834758 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:05.624477 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a97cde-fed2-46d6-8105-982f4e834758 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:05.624945 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a97cde-fed2-46d6-8105-982f4e834758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:05.635717 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:05.656595 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-93a97cde-fed2-46d6-8105-982f4e834758 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:03:05.657842 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-93a97cde-fed2-46d6-8105-982f4e834758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:03:05.699109 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ea1816-9468-4d07-b521-c245234ebfa0 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:05.744563 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 232/473] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:03:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 21:03:05.744749 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc1b6e9-61cf-4e9c-9e89-0ec12441531f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:05.745227 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc1b6e9-61cf-4e9c-9e89-0ec12441531f None admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:05.745420 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a97cde-fed2-46d6-8105-982f4e834758 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:05.745512 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc1b6e9-61cf-4e9c-9e89-0ec12441531f None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:05.745771 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc1b6e9-61cf-4e9c-9e89-0ec12441531f None admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:05.746464 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc1b6e9-61cf-4e9c-9e89-0ec12441531f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:05.748013 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc1b6e9-61cf-4e9c-9e89-0ec12441531f None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:05.808852 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:05.852152 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 242/474] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Aug 4 21:03:05 2022] POST /identity/v3/users => generated 248 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:03:05.853236 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc1b6e9-61cf-4e9c-9e89-0ec12441531f None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:05.863903 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a97cde-fed2-46d6-8105-982f4e834758 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:05.925477 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb435ac5-cb54-44e3-a654-692752bf09bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:05.927082 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb435ac5-cb54-44e3-a654-692752bf09bc None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:05.927580 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb435ac5-cb54-44e3-a654-692752bf09bc None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:05.928687 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb435ac5-cb54-44e3-a654-692752bf09bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:05.954108 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb435ac5-cb54-44e3-a654-692752bf09bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:05.958754 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb435ac5-cb54-44e3-a654-692752bf09bc None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:05.979449 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-fb435ac5-cb54-44e3-a654-692752bf09bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:05.979449 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:05.979449 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:05.979449 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:05.979449 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:05.979449 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:05.979449 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:05.979449 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:05.979449 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:05.979449 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:05.979449 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:05.979449 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:05.979449 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:03:05.979449 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:03:05.979449 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:03:05.981758 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:03:05.981758 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:05.981758 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:05.981758 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:05.981758 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:05.981758 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:05.981758 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:05.981758 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:05.981758 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:05.981758 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:05.981758 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:05.981758 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:05.981758 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:05.981758 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:05.981758 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:05.981758 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:05.981758 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:05.982629 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:03:05.982629 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:03:05.982629 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:05.982629 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:05.986101 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 233/475] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:03:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:05.988114 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb435ac5-cb54-44e3-a654-692752bf09bc None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:06.069124 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc1b6e9-61cf-4e9c-9e89-0ec12441531f None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:06.076431 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:06.103999 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb435ac5-cb54-44e3-a654-692752bf09bc None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:06.106545 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b4fa52-0e70-4647-ac1e-4edc3d06b693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:06.107347 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b4fa52-0e70-4647-ac1e-4edc3d06b693 None admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:06.107853 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b4fa52-0e70-4647-ac1e-4edc3d06b693 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:06.107966 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b4fa52-0e70-4647-ac1e-4edc3d06b693 None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:06.108702 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b4fa52-0e70-4647-ac1e-4edc3d06b693 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:06.110550 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b4fa52-0e70-4647-ac1e-4edc3d06b693 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:06.117784 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:06.143690 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.sql.core [None req-d2b4fa52-0e70-4647-ac1e-4edc3d06b693 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 04 21:03:06.143690 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 04 21:03:06.143690 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [parameters: {'id': 'ed39aa20c083487db8b272f10697f030', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 04 21:03:06.143690 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80601) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 04 21:03:06.146588 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1ea82d-8ba8-4d9b-9442-494a24c1b468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:06.147270 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ea82d-8ba8-4d9b-9442-494a24c1b468 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:06.147624 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ea82d-8ba8-4d9b-9442-494a24c1b468 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:06.147880 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ea82d-8ba8-4d9b-9442-494a24c1b468 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:06.148583 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1ea82d-8ba8-4d9b-9442-494a24c1b468 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:06.148889 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1ea82d-8ba8-4d9b-9442-494a24c1b468 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:06.155048 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1ea82d-8ba8-4d9b-9442-494a24c1b468 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:06.163986 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-d2b4fa52-0e70-4647-ac1e-4edc3d06b693 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 04 21:03:06.163986 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:06.163986 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 04 21:03:06.163986 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 04 21:03:06.163986 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 04 21:03:06.163986 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 04 21:03:06.163986 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 04 21:03:06.163986 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application result = self._query(query) Aug 04 21:03:06.163986 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 04 21:03:06.163986 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application conn.query(q) Aug 04 21:03:06.163986 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 04 21:03:06.163986 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 04 21:03:06.163986 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 04 21:03:06.163986 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application result.read() Aug 04 21:03:06.163986 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 04 21:03:06.164938 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 04 21:03:06.164938 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 04 21:03:06.164938 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 04 21:03:06.164938 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 04 21:03:06.164938 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 04 21:03:06.164938 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 04 21:03:06.164938 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 04 21:03:06.164938 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 04 21:03:06.164938 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:06.164938 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 04 21:03:06.164938 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:06.164938 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:06.164938 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 04 21:03:06.164938 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 04 21:03:06.164938 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 04 21:03:06.164938 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return ref.to_dict() Aug 04 21:03:06.164938 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 04 21:03:06.164938 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application next(self.gen) Aug 04 21:03:06.165830 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 04 21:03:06.165830 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application yield resource Aug 04 21:03:06.165830 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 04 21:03:06.165830 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application next(self.gen) Aug 04 21:03:06.165830 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 04 21:03:06.165830 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application self.session.rollback() Aug 04 21:03:06.165830 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 04 21:03:06.165830 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application self.force_reraise() Aug 04 21:03:06.165830 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 04 21:03:06.165830 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise self.value Aug 04 21:03:06.165830 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 04 21:03:06.165830 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 04 21:03:06.165830 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 04 21:03:06.165830 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application session.commit() Aug 04 21:03:06.165830 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 04 21:03:06.165830 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 04 21:03:06.165830 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 04 21:03:06.166668 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application self._prepare_impl() Aug 04 21:03:06.166668 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 04 21:03:06.166668 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application self.session.flush() Aug 04 21:03:06.166668 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 04 21:03:06.166668 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application self._flush(objects) Aug 04 21:03:06.166668 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 04 21:03:06.166668 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 04 21:03:06.166668 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 04 21:03:06.166668 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application compat.raise_( Aug 04 21:03:06.166668 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 04 21:03:06.166668 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception Aug 04 21:03:06.166668 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 04 21:03:06.166668 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application flush_context.execute() Aug 04 21:03:06.166668 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 04 21:03:06.166668 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rec.execute(self) Aug 04 21:03:06.166668 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 04 21:03:06.166668 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 04 21:03:06.167558 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 04 21:03:06.167558 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 04 21:03:06.167558 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 04 21:03:06.167558 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 04 21:03:06.167558 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 04 21:03:06.167558 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 04 21:03:06.167558 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 04 21:03:06.167558 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 04 21:03:06.167558 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 04 21:03:06.167558 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 04 21:03:06.167558 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 04 21:03:06.167558 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 04 21:03:06.167558 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 04 21:03:06.167558 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 04 21:03:06.167558 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 04 21:03:06.167558 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception Aug 04 21:03:06.175220 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 04 21:03:06.175220 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 04 21:03:06.175220 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 04 21:03:06.175220 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 04 21:03:06.175220 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 04 21:03:06.175220 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application result = self._query(query) Aug 04 21:03:06.175220 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 04 21:03:06.175220 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application conn.query(q) Aug 04 21:03:06.175220 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 04 21:03:06.175220 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 04 21:03:06.175220 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 04 21:03:06.175220 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application result.read() Aug 04 21:03:06.175220 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 04 21:03:06.175220 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 04 21:03:06.175220 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 04 21:03:06.175220 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 04 21:03:06.184669 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 04 21:03:06.184669 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 04 21:03:06.184669 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 04 21:03:06.184669 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 04 21:03:06.184669 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 04 21:03:06.184669 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 04 21:03:06.184669 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [parameters: {'id': 'ed39aa20c083487db8b272f10697f030', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 04 21:03:06.184669 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 04 21:03:06.184669 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:06.184669 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 21:03:06.184669 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:06.184669 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:06.184669 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:06.184669 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:06.184669 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:06.184669 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:06.185501 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:06.185501 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:06.185501 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:06.185501 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:06.185501 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:06.185501 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:06.185501 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 04 21:03:06.185501 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 04 21:03:06.185501 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:06.185501 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:06.185501 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 04 21:03:06.185501 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 04 21:03:06.185501 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 04 21:03:06.185501 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 04 21:03:06.185501 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 04 21:03:06.185501 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:06.186325 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 243/477] 10.222.0.25 () {60 vars in 1131 bytes} [Thu Aug 4 21:03:05 2022] POST /identity/v3/roles => generated 144 bytes in 313 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:06.186389 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 234/477] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:03:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:06.186465 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b4fa52-0e70-4647-ac1e-4edc3d06b693 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:06.186531 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1ea82d-8ba8-4d9b-9442-494a24c1b468 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:06.310058 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1ea82d-8ba8-4d9b-9442-494a24c1b468 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:06.320447 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b4fa52-0e70-4647-ac1e-4edc3d06b693 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:06.353297 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f3ef16-2a78-4b88-bbe3-ac0510bb2db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:06.354282 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f3ef16-2a78-4b88-bbe3-ac0510bb2db1 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:06.354999 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f3ef16-2a78-4b88-bbe3-ac0510bb2db1 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:06.355551 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f3ef16-2a78-4b88-bbe3-ac0510bb2db1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:06.365761 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa59b3b-6c96-4fd7-b587-11ef1f97f5b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:06.366406 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa59b3b-6c96-4fd7-b587-11ef1f97f5b4 None admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:06.366742 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa59b3b-6c96-4fd7-b587-11ef1f97f5b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:06.367022 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa59b3b-6c96-4fd7-b587-11ef1f97f5b4 None admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:06.367752 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa59b3b-6c96-4fd7-b587-11ef1f97f5b4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:06.369282 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa59b3b-6c96-4fd7-b587-11ef1f97f5b4 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:06.373546 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f3ef16-2a78-4b88-bbe3-ac0510bb2db1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:03:06.379078 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-29f3ef16-2a78-4b88-bbe3-ac0510bb2db1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 21:03:06.379078 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:06.379078 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:06.379078 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:06.379078 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:06.379078 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:06.379078 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:06.379078 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:06.379078 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:06.379078 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:06.379078 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:06.379078 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:06.379078 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:03:06.379078 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:03:06.379078 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:03:06.379078 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:03:06.381091 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:03:06.381091 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:03:06.381091 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:03:06.381091 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:03:06.381091 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:06.381091 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:06.381091 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:06.381091 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:06.381091 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:06.381091 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:06.381091 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:06.381091 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:06.381091 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:06.381091 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:06.381091 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:06.381091 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:06.381867 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:06.381867 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:06.381867 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:06.381867 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:06.381867 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:03:06.381867 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:03:06.381867 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:03:06.381867 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:03:06.381867 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:03:06.381867 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:03:06.381867 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 21:03:06.381867 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:06.385079 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 235/478] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Aug 4 21:03:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:03:06.390822 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f3ef16-2a78-4b88-bbe3-ac0510bb2db1 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:06.425805 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:06.470407 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 244/479] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Aug 4 21:03:06 2022] POST /identity/v3/users => generated 249 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:03:06.506691 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f3ef16-2a78-4b88-bbe3-ac0510bb2db1 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:06.545426 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bf867f-7ec1-4b3c-9acc-271ed010090d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:06.546055 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bf867f-7ec1-4b3c-9acc-271ed010090d None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:06.546336 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bf867f-7ec1-4b3c-9acc-271ed010090d None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:06.546605 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bf867f-7ec1-4b3c-9acc-271ed010090d None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:06.547338 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bf867f-7ec1-4b3c-9acc-271ed010090d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:06.547653 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bf867f-7ec1-4b3c-9acc-271ed010090d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:06.549042 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bf867f-7ec1-4b3c-9acc-271ed010090d None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:06.558442 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 236/480] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Aug 4 21:03:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:03:07.969127 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201bd218-9541-48dd-aa4b-6e125b2829d1 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:07.969648 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201bd218-9541-48dd-aa4b-6e125b2829d1 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:07.969648 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201bd218-9541-48dd-aa4b-6e125b2829d1 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:07.972345 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 245/481] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:03:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:03:07.978768 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfda8c5-4621-40bc-939b-19a9b02f14d3 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:07.979246 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfda8c5-4621-40bc-939b-19a9b02f14d3 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:07.979352 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfda8c5-4621-40bc-939b-19a9b02f14d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:08.001680 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-bbfda8c5-4621-40bc-939b-19a9b02f14d3 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:03:08.003126 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-bbfda8c5-4621-40bc-939b-19a9b02f14d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:03:08.073907 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57184867-a93e-4a04-b291-6c6e97d5ba92 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:08.074240 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57184867-a93e-4a04-b291-6c6e97d5ba92 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:08.074461 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57184867-a93e-4a04-b291-6c6e97d5ba92 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:08.076105 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 246/482] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:03:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:03:08.081645 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c87931-9387-47f5-af3b-4dba10f527e0 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:08.082198 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c87931-9387-47f5-af3b-4dba10f527e0 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:08.082672 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c87931-9387-47f5-af3b-4dba10f527e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:08.090869 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 237/483] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:03:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:03:08.106530 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-95c87931-9387-47f5-af3b-4dba10f527e0 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:03:08.108138 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-95c87931-9387-47f5-af3b-4dba10f527e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:03:08.115671 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfda8c5-4621-40bc-939b-19a9b02f14d3 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:08.138059 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:08.156257 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:08.231530 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 247/484] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:03:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:03:08.265920 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c87931-9387-47f5-af3b-4dba10f527e0 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:08.288883 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfda8c5-4621-40bc-939b-19a9b02f14d3 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:08.341499 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c607c0c2-bdd4-487c-a243-83bfebdd1bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:08.342488 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c607c0c2-bdd4-487c-a243-83bfebdd1bc9 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:08.342922 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c607c0c2-bdd4-487c-a243-83bfebdd1bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:08.343366 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c607c0c2-bdd4-487c-a243-83bfebdd1bc9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:08.365338 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c607c0c2-bdd4-487c-a243-83bfebdd1bc9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:03:08.370774 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-c607c0c2-bdd4-487c-a243-83bfebdd1bc9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:03:08.370774 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:08.370774 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:08.370774 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:08.370774 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:08.370774 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:08.370774 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:08.370774 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:08.370774 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:08.370774 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:08.370774 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:08.370774 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:08.370774 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:03:08.370774 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:03:08.370774 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:03:08.370774 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:03:08.371795 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:03:08.371795 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:03:08.371795 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:03:08.371795 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:03:08.371795 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:08.371795 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:08.371795 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:08.371795 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:08.371795 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:08.371795 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:08.371795 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:08.371795 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:08.371795 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:08.371795 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:08.371795 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:08.371795 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:08.372625 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:08.372625 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:08.372625 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:08.372625 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:08.372625 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:03:08.372625 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:03:08.372625 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:03:08.372625 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:03:08.372625 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:03:08.372625 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:03:08.372625 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:03:08.372625 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:08.375178 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 238/485] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Aug 4 21:03:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:03:08.382244 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c607c0c2-bdd4-487c-a243-83bfebdd1bc9 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:08.466158 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c87931-9387-47f5-af3b-4dba10f527e0 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:08.526138 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a50be7d-5793-4917-a45b-c0d081f4137e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:08.526710 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a50be7d-5793-4917-a45b-c0d081f4137e None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:08.526991 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a50be7d-5793-4917-a45b-c0d081f4137e None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:08.527270 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a50be7d-5793-4917-a45b-c0d081f4137e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:08.536373 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:08.542665 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a50be7d-5793-4917-a45b-c0d081f4137e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:03:08.549959 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-8a50be7d-5793-4917-a45b-c0d081f4137e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:03:08.549959 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:08.549959 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:08.549959 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:08.549959 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:08.549959 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:08.549959 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:08.549959 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:08.549959 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:08.549959 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:08.549959 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:08.549959 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:08.549959 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:03:08.549959 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:03:08.549959 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:03:08.549959 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:03:08.551521 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:03:08.551521 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:03:08.551521 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:03:08.551521 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:03:08.551521 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:08.551521 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:08.551521 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:08.551521 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:08.551521 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:08.551521 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:08.551521 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:08.551521 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:08.551521 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:08.551521 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:08.551521 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:08.551521 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:08.552400 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:08.552400 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:08.552400 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:08.552400 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:08.552400 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:03:08.552400 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:03:08.552400 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:03:08.552400 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:03:08.552400 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:03:08.552400 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:03:08.552400 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:03:08.552400 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:08.555667 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 248/486] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Aug 4 21:03:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:03:08.558246 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e91210-bd25-45ca-a005-fe5529c2dd23 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:08.558531 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e91210-bd25-45ca-a005-fe5529c2dd23 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:08.558806 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e91210-bd25-45ca-a005-fe5529c2dd23 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:08.560440 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 249/487] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:03:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:03:08.561098 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e91210-bd25-45ca-a005-fe5529c2dd23 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:08.563739 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c607c0c2-bdd4-487c-a243-83bfebdd1bc9 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:08.588061 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:08.607484 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a71cc99-29f5-47f6-a214-48f8730367ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:08.608342 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a71cc99-29f5-47f6-a214-48f8730367ac None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:08.608712 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a71cc99-29f5-47f6-a214-48f8730367ac None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:08.609082 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a71cc99-29f5-47f6-a214-48f8730367ac None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:08.609911 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a71cc99-29f5-47f6-a214-48f8730367ac None admin] RBAC: Adding query filter params (name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:08.610307 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a71cc99-29f5-47f6-a214-48f8730367ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:08.612430 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a71cc99-29f5-47f6-a214-48f8730367ac None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:08.622342 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 239/488] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Aug 4 21:03:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:08.623824 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd59c02-2678-4c4c-ad82-a94c3dc59aa7 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:08.624244 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd59c02-2678-4c4c-ad82-a94c3dc59aa7 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:08.624634 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd59c02-2678-4c4c-ad82-a94c3dc59aa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:08.644228 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-7fd59c02-2678-4c4c-ad82-a94c3dc59aa7 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:03:08.645592 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-7fd59c02-2678-4c4c-ad82-a94c3dc59aa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:03:08.698521 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e91210-bd25-45ca-a005-fe5529c2dd23 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:08.734338 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f1fe7f-506c-4d4a-aeaf-bf0d3c88f3f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:08.735135 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f1fe7f-506c-4d4a-aeaf-bf0d3c88f3f9 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:08.735538 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f1fe7f-506c-4d4a-aeaf-bf0d3c88f3f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:08.735913 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f1fe7f-506c-4d4a-aeaf-bf0d3c88f3f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:08.737999 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f1fe7f-506c-4d4a-aeaf-bf0d3c88f3f9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:08.743361 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f1fe7f-506c-4d4a-aeaf-bf0d3c88f3f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:08.743361 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f1fe7f-506c-4d4a-aeaf-bf0d3c88f3f9 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:08.743746 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 240/489] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:03:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:03:08.745512 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd59c02-2678-4c4c-ad82-a94c3dc59aa7 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:08.765600 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 250/490] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Aug 4 21:03:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:08.766445 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f1fe7f-506c-4d4a-aeaf-bf0d3c88f3f9 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:08.833438 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:08.861292 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd59c02-2678-4c4c-ad82-a94c3dc59aa7 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:08.876163 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:08.906166 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f1fe7f-506c-4d4a-aeaf-bf0d3c88f3f9 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:08.908169 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec8df39-1a47-4fae-930c-92a375746534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:08.909105 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec8df39-1a47-4fae-930c-92a375746534 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:08.909383 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec8df39-1a47-4fae-930c-92a375746534 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:08.909651 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec8df39-1a47-4fae-930c-92a375746534 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:08.928876 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec8df39-1a47-4fae-930c-92a375746534 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:08.931273 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec8df39-1a47-4fae-930c-92a375746534 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:08.946126 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3968de4-6721-400c-9a35-42dbe374964a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:08.946766 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3968de4-6721-400c-9a35-42dbe374964a None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:08.947048 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3968de4-6721-400c-9a35-42dbe374964a None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:08.947382 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3968de4-6721-400c-9a35-42dbe374964a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:08.953028 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-fec8df39-1a47-4fae-930c-92a375746534 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:08.953028 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:08.953028 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:08.953028 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:08.953028 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:08.953028 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:08.953028 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:08.953028 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:08.953028 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:08.953028 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:08.953028 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:08.953028 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:08.953028 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:03:08.953028 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:03:08.953028 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:03:08.953858 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:03:08.953858 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:08.953858 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:08.953858 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:08.953858 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:08.953858 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:08.953858 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:08.953858 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:08.953858 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:08.953858 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:08.953858 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:08.953858 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:08.953858 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:08.953858 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:08.953858 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:08.953858 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:08.953858 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:08.954713 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:03:08.954713 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:03:08.954713 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:08.954713 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:08.957766 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 241/491] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:03:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:08.960137 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec8df39-1a47-4fae-930c-92a375746534 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:08.965862 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3968de4-6721-400c-9a35-42dbe374964a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:03:08.978310 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-b3968de4-6721-400c-9a35-42dbe374964a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:03:08.978310 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:08.978310 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:08.978310 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:08.978310 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:08.978310 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:08.978310 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:08.978310 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:08.978310 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:08.978310 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:08.978310 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:08.978310 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:08.978310 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:03:08.978310 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:03:08.978310 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:03:08.978310 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:03:08.979951 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:03:08.979951 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:03:08.979951 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:03:08.979951 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:03:08.979951 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:08.979951 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:08.979951 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:08.979951 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:08.979951 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:08.979951 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:08.979951 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:08.979951 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:08.979951 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:08.979951 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:08.979951 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:08.979951 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:08.980820 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:08.980820 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:08.980820 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:08.980820 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:08.980820 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:03:08.980820 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:03:08.980820 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:03:08.980820 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:03:08.980820 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:03:08.980820 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:03:08.980820 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:03:08.980820 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:08.987606 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 251/492] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Aug 4 21:03:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:08.987606 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3968de4-6721-400c-9a35-42dbe374964a None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:09.147642 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec8df39-1a47-4fae-930c-92a375746534 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:09.151204 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3968de4-6721-400c-9a35-42dbe374964a None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:09.173298 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:09.181699 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe395dd8-db12-496e-ae2d-d3c77b4ec30e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:09.182585 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe395dd8-db12-496e-ae2d-d3c77b4ec30e None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:09.182876 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe395dd8-db12-496e-ae2d-d3c77b4ec30e None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:09.183148 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe395dd8-db12-496e-ae2d-d3c77b4ec30e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:09.185118 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de9d663-3dd4-49ed-aef0-2ff179deabff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:09.185733 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de9d663-3dd4-49ed-aef0-2ff179deabff None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:09.186015 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de9d663-3dd4-49ed-aef0-2ff179deabff None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:09.186287 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de9d663-3dd4-49ed-aef0-2ff179deabff None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:09.186977 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de9d663-3dd4-49ed-aef0-2ff179deabff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:09.187279 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de9d663-3dd4-49ed-aef0-2ff179deabff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:09.188681 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de9d663-3dd4-49ed-aef0-2ff179deabff None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:09.194428 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe395dd8-db12-496e-ae2d-d3c77b4ec30e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:09.196031 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe395dd8-db12-496e-ae2d-d3c77b4ec30e None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:09.196267 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:09.201479 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 252/493] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:03:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:09.202254 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de9d663-3dd4-49ed-aef0-2ff179deabff None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:09.211648 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-fe395dd8-db12-496e-ae2d-d3c77b4ec30e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:09.211648 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:09.211648 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:09.211648 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:09.211648 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:09.211648 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:09.211648 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:09.211648 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:09.211648 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:09.211648 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:09.211648 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:09.211648 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:09.211648 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:03:09.211648 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:03:09.211648 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:03:09.212684 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:03:09.212684 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:09.212684 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:09.212684 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:09.212684 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:09.212684 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:09.212684 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:09.212684 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:09.212684 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:09.212684 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:09.212684 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:09.212684 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:09.212684 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:09.212684 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:09.212684 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:09.212684 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:09.212684 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:09.213523 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:03:09.213523 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:03:09.213523 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:09.213523 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:09.213523 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 242/494] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:03:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:09.213918 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592d7c24-afd2-49b5-b023-098feaa10960 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:09.214196 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592d7c24-afd2-49b5-b023-098feaa10960 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:09.214462 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592d7c24-afd2-49b5-b023-098feaa10960 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:09.215862 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 243/495] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:03:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:03:09.216758 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9c73b6-fec3-431f-acc9-d2fea97a16eb None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:09.217202 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9c73b6-fec3-431f-acc9-d2fea97a16eb None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:09.217295 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9c73b6-fec3-431f-acc9-d2fea97a16eb None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:09.222846 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 244/496] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:03:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:03:09.222846 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9c73b6-fec3-431f-acc9-d2fea97a16eb None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:09.322571 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9c73b6-fec3-431f-acc9-d2fea97a16eb None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:09.341338 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de9d663-3dd4-49ed-aef0-2ff179deabff None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:09.355673 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e51eb92-5afc-4413-81a5-36e14f623c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:09.356386 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e51eb92-5afc-4413-81a5-36e14f623c2c None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:09.356668 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e51eb92-5afc-4413-81a5-36e14f623c2c None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:09.356932 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e51eb92-5afc-4413-81a5-36e14f623c2c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:09.364796 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e51eb92-5afc-4413-81a5-36e14f623c2c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:09.366423 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e51eb92-5afc-4413-81a5-36e14f623c2c None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:09.379854 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-2e51eb92-5afc-4413-81a5-36e14f623c2c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 21:03:09.379854 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:09.379854 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:09.379854 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:09.379854 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:09.379854 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:09.379854 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:09.379854 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:09.379854 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:09.379854 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:09.379854 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:09.379854 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:09.379854 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 21:03:09.379854 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 21:03:09.379854 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 21:03:09.379854 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 21:03:09.380793 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:09.380793 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:09.380793 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 21:03:09.380793 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:03:09.380793 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 21:03:09.380793 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:03:09.380793 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:09.380793 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:09.380793 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:09.380793 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:09.380793 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:09.380793 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:09.380793 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:09.380793 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:09.380793 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:09.380793 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:09.380793 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:09.380793 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:09.381676 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:09.381676 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:09.381676 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 21:03:09.381676 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 21:03:09.381676 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 21:03:09.381676 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 21:03:09.381676 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 21:03:09.381676 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 21:03:09.381676 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 21:03:09.381676 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:09.382184 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25e5e39-bef1-4f11-b8dc-835b5c1ff553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:09.382277 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 245/497] 10.222.0.25 () {56 vars in 1093 bytes} [Thu Aug 4 21:03:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:09.382344 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e5e39-bef1-4f11-b8dc-835b5c1ff553 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:09.382344 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e5e39-bef1-4f11-b8dc-835b5c1ff553 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:09.382471 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e51eb92-5afc-4413-81a5-36e14f623c2c None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:09.382535 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e5e39-bef1-4f11-b8dc-835b5c1ff553 None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:09.383062 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25e5e39-bef1-4f11-b8dc-835b5c1ff553 None admin] RBAC: Adding query filter params (name=service) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:09.383398 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25e5e39-bef1-4f11-b8dc-835b5c1ff553 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:09.384821 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25e5e39-bef1-4f11-b8dc-835b5c1ff553 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:09.396751 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 253/498] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Aug 4 21:03:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:09.398937 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bbb5ce-1ab6-417c-8530-f96b799eff7d None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:09.399333 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bbb5ce-1ab6-417c-8530-f96b799eff7d None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:09.399750 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bbb5ce-1ab6-417c-8530-f96b799eff7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:09.425222 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-57bbb5ce-1ab6-417c-8530-f96b799eff7d None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:03:09.426445 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-57bbb5ce-1ab6-417c-8530-f96b799eff7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:03:09.426789 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:09.481413 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:09.486852 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e51eb92-5afc-4413-81a5-36e14f623c2c None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:09.492120 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 254/499] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:03:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:03:09.493396 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06139431-1d0d-4ad1-bb14-84a95c4d2726 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:09.493740 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06139431-1d0d-4ad1-bb14-84a95c4d2726 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:09.494023 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06139431-1d0d-4ad1-bb14-84a95c4d2726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:09.511457 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-06139431-1d0d-4ad1-bb14-84a95c4d2726 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:03:09.512627 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-06139431-1d0d-4ad1-bb14-84a95c4d2726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:03:09.524864 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eea2ae-7e66-46c7-97dd-71bdab7b84a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:09.525490 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eea2ae-7e66-46c7-97dd-71bdab7b84a8 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:09.525773 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eea2ae-7e66-46c7-97dd-71bdab7b84a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:09.526051 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eea2ae-7e66-46c7-97dd-71bdab7b84a8 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:09.526749 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eea2ae-7e66-46c7-97dd-71bdab7b84a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:09.527058 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eea2ae-7e66-46c7-97dd-71bdab7b84a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:09.528498 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eea2ae-7e66-46c7-97dd-71bdab7b84a8 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:09.540563 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 246/500] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:03:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:09.541339 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eea2ae-7e66-46c7-97dd-71bdab7b84a8 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:09.583430 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 255/501] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:03:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:03:09.584444 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06139431-1d0d-4ad1-bb14-84a95c4d2726 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:09.648397 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:09.652320 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eea2ae-7e66-46c7-97dd-71bdab7b84a8 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:09.681686 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06139431-1d0d-4ad1-bb14-84a95c4d2726 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:09.687991 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da578252-a42e-4077-9d71-21c216e78f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:09.688608 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da578252-a42e-4077-9d71-21c216e78f9b None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:09.688894 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da578252-a42e-4077-9d71-21c216e78f9b None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:09.689169 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da578252-a42e-4077-9d71-21c216e78f9b None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:09.690023 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da578252-a42e-4077-9d71-21c216e78f9b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:09.690332 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da578252-a42e-4077-9d71-21c216e78f9b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:09.691822 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da578252-a42e-4077-9d71-21c216e78f9b None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:09.701272 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:09.715366 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33b8d69-3356-4f7d-b2ab-caf35e872bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:09.716002 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33b8d69-3356-4f7d-b2ab-caf35e872bc6 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:09.716281 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33b8d69-3356-4f7d-b2ab-caf35e872bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:09.716550 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33b8d69-3356-4f7d-b2ab-caf35e872bc6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:09.727177 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33b8d69-3356-4f7d-b2ab-caf35e872bc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:09.728763 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33b8d69-3356-4f7d-b2ab-caf35e872bc6 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:09.736683 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 247/502] 10.222.0.25 () {56 vars in 1133 bytes} [Thu Aug 4 21:03:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:09.736683 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da578252-a42e-4077-9d71-21c216e78f9b None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:09.745924 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-e33b8d69-3356-4f7d-b2ab-caf35e872bc6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:09.745924 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:09.745924 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:09.745924 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:09.745924 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:09.745924 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:09.745924 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:09.745924 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:09.745924 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:09.745924 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:09.745924 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:09.745924 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:09.745924 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:03:09.745924 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:03:09.745924 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:03:09.746785 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:03:09.746785 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:09.746785 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:09.746785 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:09.746785 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:09.746785 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:09.746785 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:09.746785 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:09.746785 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:09.746785 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:09.746785 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:09.746785 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:09.746785 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:09.746785 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:09.746785 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:09.746785 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:09.746785 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:09.747652 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:03:09.747652 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:03:09.747652 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:09.747652 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:09.747652 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 256/503] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:03:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:09.748191 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33b8d69-3356-4f7d-b2ab-caf35e872bc6 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:09.838912 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da578252-a42e-4077-9d71-21c216e78f9b None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:09.848510 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33b8d69-3356-4f7d-b2ab-caf35e872bc6 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:09.871672 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dd6369-00bc-46ad-8a45-44b38e5f87ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:09.872311 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd6369-00bc-46ad-8a45-44b38e5f87ab None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:09.872601 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd6369-00bc-46ad-8a45-44b38e5f87ab None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:09.872884 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd6369-00bc-46ad-8a45-44b38e5f87ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:09.881384 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759d6f54-318e-467c-85a0-dbb887bd7558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:09.882016 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759d6f54-318e-467c-85a0-dbb887bd7558 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:09.882300 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759d6f54-318e-467c-85a0-dbb887bd7558 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:09.882572 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759d6f54-318e-467c-85a0-dbb887bd7558 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:09.883192 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dd6369-00bc-46ad-8a45-44b38e5f87ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:09.884692 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dd6369-00bc-46ad-8a45-44b38e5f87ab None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:09.889935 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759d6f54-318e-467c-85a0-dbb887bd7558 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:09.891389 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759d6f54-318e-467c-85a0-dbb887bd7558 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:09.899652 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-c0dd6369-00bc-46ad-8a45-44b38e5f87ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:09.899652 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:09.899652 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:09.899652 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:09.899652 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:09.899652 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:09.899652 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:09.899652 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:09.899652 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:09.899652 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:09.899652 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:09.899652 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:09.899652 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:03:09.899652 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:03:09.899652 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:03:09.900558 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:03:09.900558 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:09.900558 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:09.900558 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:09.900558 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:09.900558 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:09.900558 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:09.900558 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:09.900558 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:09.900558 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:09.900558 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:09.900558 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:09.900558 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:09.900558 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:09.900558 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:09.900558 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:09.900558 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:09.901387 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:03:09.901387 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:03:09.901387 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:09.901387 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:09.901387 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 248/504] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:03:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:09.901653 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dd6369-00bc-46ad-8a45-44b38e5f87ab None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:09.904436 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-759d6f54-318e-467c-85a0-dbb887bd7558 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 21:03:09.904436 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:09.904436 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:09.904436 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:09.904436 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:09.904436 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:09.904436 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:09.904436 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:09.904436 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:09.904436 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:09.904436 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:09.904436 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:09.904436 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 21:03:09.904436 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 21:03:09.904436 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 21:03:09.904436 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 21:03:09.905264 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:09.905264 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:09.905264 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 21:03:09.905264 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:03:09.905264 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 21:03:09.905264 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:03:09.905264 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:09.905264 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:09.905264 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:09.905264 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:09.905264 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:09.905264 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:09.905264 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:09.905264 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:09.905264 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:09.905264 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:09.905264 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:09.905264 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:09.907305 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:09.907305 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:09.907305 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 21:03:09.907305 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 21:03:09.907305 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 21:03:09.907305 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 21:03:09.907305 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 21:03:09.907305 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 21:03:09.907305 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 21:03:09.907305 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:09.907305 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 257/505] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Aug 4 21:03:09 2022] GET /identity/v3/users/cinder => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:09.907305 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759d6f54-318e-467c-85a0-dbb887bd7558 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:09.914765 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:09.939144 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:10.001561 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dd6369-00bc-46ad-8a45-44b38e5f87ab None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:10.006010 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759d6f54-318e-467c-85a0-dbb887bd7558 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:10.035697 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730075f8-9188-42b1-be56-716c7b30f541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:10.036331 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730075f8-9188-42b1-be56-716c7b30f541 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:10.036695 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730075f8-9188-42b1-be56-716c7b30f541 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:10.036968 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730075f8-9188-42b1-be56-716c7b30f541 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:10.040973 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832cc9ef-e28e-4c8e-afcc-c770b3c44b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:10.041595 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832cc9ef-e28e-4c8e-afcc-c770b3c44b1b None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:10.041877 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832cc9ef-e28e-4c8e-afcc-c770b3c44b1b None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:10.042154 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832cc9ef-e28e-4c8e-afcc-c770b3c44b1b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:10.046649 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730075f8-9188-42b1-be56-716c7b30f541 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:03:10.051952 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-730075f8-9188-42b1-be56-716c7b30f541 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:03:10.051952 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:10.051952 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:10.051952 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:10.051952 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:10.051952 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:10.051952 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:10.051952 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:10.051952 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:10.051952 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:10.051952 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:10.051952 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:10.051952 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:03:10.051952 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:03:10.051952 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:03:10.051952 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:03:10.052999 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:03:10.052999 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:03:10.052999 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:03:10.052999 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:03:10.052999 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:10.052999 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:10.052999 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:10.052999 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:10.052999 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:10.052999 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:10.052999 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:10.052999 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:10.052999 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:10.052999 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:10.052999 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:10.052999 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:10.053999 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:10.053999 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:10.053999 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:10.053999 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:10.053999 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:03:10.053999 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:03:10.053999 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:03:10.053999 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:03:10.053999 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:03:10.053999 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:03:10.053999 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:03:10.053999 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:10.053999 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 249/506] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Aug 4 21:03:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:10.053999 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730075f8-9188-42b1-be56-716c7b30f541 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:10.113232 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832cc9ef-e28e-4c8e-afcc-c770b3c44b1b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:10.114721 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832cc9ef-e28e-4c8e-afcc-c770b3c44b1b None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:10.120038 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:10.144398 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:10.152917 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730075f8-9188-42b1-be56-716c7b30f541 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:10.189452 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fd4db7-6bbf-459b-b170-c609fdd05996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:10.190661 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fd4db7-6bbf-459b-b170-c609fdd05996 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:10.190938 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fd4db7-6bbf-459b-b170-c609fdd05996 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:10.191208 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fd4db7-6bbf-459b-b170-c609fdd05996 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:10.191925 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fd4db7-6bbf-459b-b170-c609fdd05996 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:10.192222 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fd4db7-6bbf-459b-b170-c609fdd05996 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:10.193644 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fd4db7-6bbf-459b-b170-c609fdd05996 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:10.204483 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 250/507] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:03:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:10.209009 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fd4db7-6bbf-459b-b170-c609fdd05996 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:10.215244 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 258/508] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Aug 4 21:03:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 21:03:10.216101 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832cc9ef-e28e-4c8e-afcc-c770b3c44b1b None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:10.306024 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fd4db7-6bbf-459b-b170-c609fdd05996 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:10.313941 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832cc9ef-e28e-4c8e-afcc-c770b3c44b1b None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:10.331331 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:10.338795 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ebeabe-7f82-4c74-be13-0269c9aae849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:10.339506 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ebeabe-7f82-4c74-be13-0269c9aae849 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:10.339788 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ebeabe-7f82-4c74-be13-0269c9aae849 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:10.343367 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ebeabe-7f82-4c74-be13-0269c9aae849 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:10.343367 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ebeabe-7f82-4c74-be13-0269c9aae849 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:10.343367 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ebeabe-7f82-4c74-be13-0269c9aae849 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:10.343367 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ebeabe-7f82-4c74-be13-0269c9aae849 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:10.343669 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:10.348383 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebb2271-7dc1-4785-98fd-31ededfb4dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:10.348999 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebb2271-7dc1-4785-98fd-31ededfb4dfb None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:10.349284 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebb2271-7dc1-4785-98fd-31ededfb4dfb None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:10.349571 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebb2271-7dc1-4785-98fd-31ededfb4dfb None admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:10.350365 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebb2271-7dc1-4785-98fd-31ededfb4dfb None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:10.350677 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebb2271-7dc1-4785-98fd-31ededfb4dfb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:10.352202 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebb2271-7dc1-4785-98fd-31ededfb4dfb None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:10.354262 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 251/509] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:03:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:10.355041 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ebeabe-7f82-4c74-be13-0269c9aae849 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:10.389515 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 259/510] 10.222.0.25 () {56 vars in 1137 bytes} [Thu Aug 4 21:03:10 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:10.390465 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebb2271-7dc1-4785-98fd-31ededfb4dfb None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:10.453424 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ebeabe-7f82-4c74-be13-0269c9aae849 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:10.487273 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5210f56-8202-4e47-96de-6f760e2105af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:10.487927 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5210f56-8202-4e47-96de-6f760e2105af None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:10.488028 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebb2271-7dc1-4785-98fd-31ededfb4dfb None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:10.488228 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5210f56-8202-4e47-96de-6f760e2105af None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:10.488478 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5210f56-8202-4e47-96de-6f760e2105af None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:10.489187 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5210f56-8202-4e47-96de-6f760e2105af None admin] RBAC: Adding query filter params (name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:10.489491 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5210f56-8202-4e47-96de-6f760e2105af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:10.490886 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5210f56-8202-4e47-96de-6f760e2105af None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:10.498615 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 252/511] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Aug 4 21:03:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:10.499345 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5210f56-8202-4e47-96de-6f760e2105af None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:10.523536 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6363c0-45e4-464a-9fe8-fc3d277044e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:10.524176 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6363c0-45e4-464a-9fe8-fc3d277044e9 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:10.524463 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6363c0-45e4-464a-9fe8-fc3d277044e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:10.524736 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6363c0-45e4-464a-9fe8-fc3d277044e9 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:10.531673 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6363c0-45e4-464a-9fe8-fc3d277044e9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:10.533122 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6363c0-45e4-464a-9fe8-fc3d277044e9 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:10.544791 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-7e6363c0-45e4-464a-9fe8-fc3d277044e9 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 04 21:03:10.544791 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:10.544791 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:10.544791 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:10.544791 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:10.544791 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:10.544791 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:10.544791 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:10.544791 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:10.544791 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:10.544791 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:10.544791 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:10.544791 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 21:03:10.544791 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 21:03:10.544791 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 21:03:10.544791 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 21:03:10.545754 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:10.545754 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:10.545754 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 21:03:10.545754 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:03:10.545754 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 21:03:10.545754 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:03:10.545754 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:10.545754 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:10.545754 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:10.545754 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:10.545754 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:10.545754 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:10.545754 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:10.545754 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:10.545754 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:10.545754 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:10.545754 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:10.545754 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:10.546651 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:10.546651 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:10.546651 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 21:03:10.546651 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 21:03:10.546651 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 21:03:10.546651 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 21:03:10.546651 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 21:03:10.546651 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 21:03:10.546651 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 04 21:03:10.546651 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:10.546651 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 260/512] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Aug 4 21:03:10 2022] GET /identity/v3/users/glance => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:10.547206 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6363c0-45e4-464a-9fe8-fc3d277044e9 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:10.549372 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:10.598033 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5210f56-8202-4e47-96de-6f760e2105af None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:10.601390 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:10.631198 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b0c2c6-89ab-4fb4-9fcc-d316a6973591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:10.631864 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b0c2c6-89ab-4fb4-9fcc-d316a6973591 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:10.632145 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b0c2c6-89ab-4fb4-9fcc-d316a6973591 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:10.632423 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b0c2c6-89ab-4fb4-9fcc-d316a6973591 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:10.642606 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b0c2c6-89ab-4fb4-9fcc-d316a6973591 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:10.644097 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b0c2c6-89ab-4fb4-9fcc-d316a6973591 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:10.650581 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6363c0-45e4-464a-9fe8-fc3d277044e9 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:10.657872 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-b4b0c2c6-89ab-4fb4-9fcc-d316a6973591 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:10.657872 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:10.657872 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:10.657872 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:10.657872 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:10.657872 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:10.657872 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:10.657872 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:10.657872 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:10.657872 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:10.657872 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:10.657872 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:10.657872 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:03:10.657872 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:03:10.657872 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:03:10.659444 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:03:10.659444 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:10.659444 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:10.659444 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:10.659444 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:10.659444 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:10.659444 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:10.659444 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:10.659444 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:10.659444 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:10.659444 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:10.659444 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:10.659444 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:10.659444 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:10.659444 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:10.659444 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:10.659444 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:10.660348 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:03:10.660348 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:03:10.660348 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:10.660348 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:10.660348 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 253/513] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:03:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:10.660348 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b0c2c6-89ab-4fb4-9fcc-d316a6973591 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:10.686950 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8292e874-b85c-47e4-a99a-e058cb5c4898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:10.687602 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8292e874-b85c-47e4-a99a-e058cb5c4898 None admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:10.687881 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8292e874-b85c-47e4-a99a-e058cb5c4898 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:10.688154 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8292e874-b85c-47e4-a99a-e058cb5c4898 None admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:10.688834 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8292e874-b85c-47e4-a99a-e058cb5c4898 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:10.690309 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8292e874-b85c-47e4-a99a-e058cb5c4898 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:10.757970 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b0c2c6-89ab-4fb4-9fcc-d316a6973591 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:10.759409 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:10.770685 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 261/514] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Aug 4 21:03:10 2022] POST /identity/v3/users => generated 247 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:03:10.771496 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8292e874-b85c-47e4-a99a-e058cb5c4898 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:10.793808 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6571d5d-4708-4258-9f14-9da1edf4b012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:10.794460 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6571d5d-4708-4258-9f14-9da1edf4b012 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:10.794906 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6571d5d-4708-4258-9f14-9da1edf4b012 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:10.795231 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6571d5d-4708-4258-9f14-9da1edf4b012 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:10.874363 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6571d5d-4708-4258-9f14-9da1edf4b012 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:10.875947 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6571d5d-4708-4258-9f14-9da1edf4b012 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:10.890445 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8292e874-b85c-47e4-a99a-e058cb5c4898 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:10.899444 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:10.929980 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e917e4-5f3a-4ccb-b2bd-c1dfc9e4b587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:10.931872 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e917e4-5f3a-4ccb-b2bd-c1dfc9e4b587 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:10.931872 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e917e4-5f3a-4ccb-b2bd-c1dfc9e4b587 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:10.932059 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e917e4-5f3a-4ccb-b2bd-c1dfc9e4b587 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:10.942675 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e917e4-5f3a-4ccb-b2bd-c1dfc9e4b587 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:10.944446 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e917e4-5f3a-4ccb-b2bd-c1dfc9e4b587 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:10.960783 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-01e917e4-5f3a-4ccb-b2bd-c1dfc9e4b587 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:10.960783 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:10.960783 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:10.960783 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:10.960783 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:10.960783 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:10.960783 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:10.960783 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:10.960783 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:10.960783 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:10.960783 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:10.960783 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:10.960783 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:03:10.960783 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:03:10.960783 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:03:10.961670 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:03:10.961670 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:10.961670 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:10.961670 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:10.961670 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:10.961670 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:10.961670 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:10.961670 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:10.961670 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:10.961670 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:10.961670 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:10.961670 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:10.961670 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:10.961670 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:10.961670 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:10.961670 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:10.961670 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:10.962516 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:03:10.962516 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:03:10.962516 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:10.962516 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:10.964431 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 262/515] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:03:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:10.966199 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e917e4-5f3a-4ccb-b2bd-c1dfc9e4b587 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:10.997837 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 254/516] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Aug 4 21:03:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 21:03:11.003994 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6571d5d-4708-4258-9f14-9da1edf4b012 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:11.006678 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:11.085895 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e917e4-5f3a-4ccb-b2bd-c1dfc9e4b587 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:11.112834 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6571d5d-4708-4258-9f14-9da1edf4b012 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:11.121612 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2510d9a4-0d08-4291-8b9f-b423e03ad64d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:11.122240 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2510d9a4-0d08-4291-8b9f-b423e03ad64d None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:11.122533 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2510d9a4-0d08-4291-8b9f-b423e03ad64d None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:11.122812 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2510d9a4-0d08-4291-8b9f-b423e03ad64d None admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:11.123649 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2510d9a4-0d08-4291-8b9f-b423e03ad64d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:11.123964 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2510d9a4-0d08-4291-8b9f-b423e03ad64d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:11.125406 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2510d9a4-0d08-4291-8b9f-b423e03ad64d None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:11.143851 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 263/517] 10.222.0.25 () {56 vars in 1137 bytes} [Thu Aug 4 21:03:10 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:11.148406 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2510d9a4-0d08-4291-8b9f-b423e03ad64d None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:11.155778 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a99998c-6f54-4bdc-b98b-48441d05e786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:11.156406 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a99998c-6f54-4bdc-b98b-48441d05e786 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:11.156692 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a99998c-6f54-4bdc-b98b-48441d05e786 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:11.156967 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a99998c-6f54-4bdc-b98b-48441d05e786 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:11.157656 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a99998c-6f54-4bdc-b98b-48441d05e786 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:11.157959 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a99998c-6f54-4bdc-b98b-48441d05e786 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:11.159896 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a99998c-6f54-4bdc-b98b-48441d05e786 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:11.165408 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:11.172960 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 255/518] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:03:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:11.173694 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a99998c-6f54-4bdc-b98b-48441d05e786 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:11.244154 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:11.264272 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2510d9a4-0d08-4291-8b9f-b423e03ad64d None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:11.285655 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a99998c-6f54-4bdc-b98b-48441d05e786 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:11.304159 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a07f8e7-a5b7-4fa5-8431-596f3ea6048e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:11.305056 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a07f8e7-a5b7-4fa5-8431-596f3ea6048e None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:11.305523 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a07f8e7-a5b7-4fa5-8431-596f3ea6048e None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:11.305978 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a07f8e7-a5b7-4fa5-8431-596f3ea6048e None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:11.306838 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a07f8e7-a5b7-4fa5-8431-596f3ea6048e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:11.307291 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a07f8e7-a5b7-4fa5-8431-596f3ea6048e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:11.308826 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a07f8e7-a5b7-4fa5-8431-596f3ea6048e None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:11.320748 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4345b6a4-064a-414e-b8c8-35ab931fb44e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:11.321389 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4345b6a4-064a-414e-b8c8-35ab931fb44e None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:11.321673 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4345b6a4-064a-414e-b8c8-35ab931fb44e None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:11.321950 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4345b6a4-064a-414e-b8c8-35ab931fb44e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:11.326555 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 264/519] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:03:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:11.327677 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a07f8e7-a5b7-4fa5-8431-596f3ea6048e None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:11.340184 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4345b6a4-064a-414e-b8c8-35ab931fb44e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:11.341620 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4345b6a4-064a-414e-b8c8-35ab931fb44e None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:11.358154 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-4345b6a4-064a-414e-b8c8-35ab931fb44e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:11.358154 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:11.358154 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:11.358154 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:11.358154 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:11.358154 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:11.358154 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:11.358154 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:11.358154 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:11.358154 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:11.358154 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:11.358154 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:11.358154 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:03:11.358154 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:03:11.358154 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:03:11.359023 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:03:11.359023 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:11.359023 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:11.359023 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:11.359023 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:11.359023 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:11.359023 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:11.359023 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:11.359023 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:11.359023 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:11.359023 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:11.359023 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:11.359023 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:11.359023 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:11.359023 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:11.359023 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:11.359023 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:11.359901 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:03:11.359901 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:03:11.359901 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:11.359901 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:11.359901 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 256/520] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:03:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:11.360187 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4345b6a4-064a-414e-b8c8-35ab931fb44e None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:11.416557 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:11.439442 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a07f8e7-a5b7-4fa5-8431-596f3ea6048e None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:11.465873 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4345b6a4-064a-414e-b8c8-35ab931fb44e None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:11.477336 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c0c975-d97b-4674-8661-540d090c2c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:11.477961 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c0c975-d97b-4674-8661-540d090c2c9f None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:11.478247 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c0c975-d97b-4674-8661-540d090c2c9f None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:11.478523 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c0c975-d97b-4674-8661-540d090c2c9f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:11.481788 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:11.503672 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf2a26a-6f27-438f-b5bc-baa8dadc2a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:11.504321 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf2a26a-6f27-438f-b5bc-baa8dadc2a8c None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:11.504608 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf2a26a-6f27-438f-b5bc-baa8dadc2a8c None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:11.505153 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf2a26a-6f27-438f-b5bc-baa8dadc2a8c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:11.512252 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf2a26a-6f27-438f-b5bc-baa8dadc2a8c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:11.513740 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf2a26a-6f27-438f-b5bc-baa8dadc2a8c None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:11.531064 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-fdf2a26a-6f27-438f-b5bc-baa8dadc2a8c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:03:11.531064 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:11.531064 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:11.531064 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:11.531064 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:11.531064 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:11.531064 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:11.531064 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:11.531064 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:11.531064 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:11.531064 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:11.531064 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:11.531064 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 21:03:11.531064 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 21:03:11.531064 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 21:03:11.532152 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 21:03:11.532152 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:11.532152 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:11.532152 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:11.532152 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:11.532152 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:11.532152 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:11.532152 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:11.532152 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:11.532152 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:11.532152 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:11.532152 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:11.532152 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:11.532152 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:11.532152 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:11.532152 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:11.532152 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:11.538747 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 21:03:11.538747 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 21:03:11.538747 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 21:03:11.538747 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 21:03:11.538747 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 21:03:11.538747 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 21:03:11.538747 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:03:11.538747 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:11.538747 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 257/521] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Aug 4 21:03:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:11.538747 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf2a26a-6f27-438f-b5bc-baa8dadc2a8c None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:11.548524 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c0c975-d97b-4674-8661-540d090c2c9f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:11.550627 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c0c975-d97b-4674-8661-540d090c2c9f None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:11.628358 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:11.651157 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf2a26a-6f27-438f-b5bc-baa8dadc2a8c None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:11.653897 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 265/522] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Aug 4 21:03:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 21:03:11.660076 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c0c975-d97b-4674-8661-540d090c2c9f None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:11.685452 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46e6252-24a9-4b9e-bd6b-0bac79f6dff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:11.686090 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46e6252-24a9-4b9e-bd6b-0bac79f6dff5 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:11.686372 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46e6252-24a9-4b9e-bd6b-0bac79f6dff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:11.686640 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46e6252-24a9-4b9e-bd6b-0bac79f6dff5 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:11.693929 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46e6252-24a9-4b9e-bd6b-0bac79f6dff5 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:11.695478 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46e6252-24a9-4b9e-bd6b-0bac79f6dff5 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:11.707605 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-a46e6252-24a9-4b9e-bd6b-0bac79f6dff5 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 04 21:03:11.707605 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:11.707605 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:11.707605 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:11.707605 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:11.707605 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:11.707605 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:11.707605 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:11.707605 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:11.707605 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:11.707605 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:11.707605 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:11.707605 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 21:03:11.707605 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 21:03:11.707605 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 21:03:11.707605 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 21:03:11.708505 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:11.708505 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:11.708505 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 21:03:11.708505 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:03:11.708505 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 21:03:11.708505 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:03:11.708505 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:11.708505 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:11.708505 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:11.708505 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:11.708505 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:11.708505 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:11.708505 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:11.708505 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:11.708505 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:11.708505 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:11.708505 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:11.708505 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:11.709415 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:11.709415 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:11.709415 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 21:03:11.709415 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 21:03:11.709415 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 21:03:11.709415 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 21:03:11.709415 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 21:03:11.709415 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 21:03:11.709415 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 04 21:03:11.709415 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:11.709415 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 258/523] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Aug 4 21:03:11 2022] GET /identity/v3/users/neutron => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:11.709969 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46e6252-24a9-4b9e-bd6b-0bac79f6dff5 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:11.747077 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:11.770647 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c0c975-d97b-4674-8661-540d090c2c9f None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:11.807483 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05e97f4-739f-48e4-a905-76db17a04058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:11.808114 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05e97f4-739f-48e4-a905-76db17a04058 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:11.808394 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05e97f4-739f-48e4-a905-76db17a04058 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:11.808667 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05e97f4-739f-48e4-a905-76db17a04058 None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:11.809423 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05e97f4-739f-48e4-a905-76db17a04058 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:11.809726 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05e97f4-739f-48e4-a905-76db17a04058 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:11.811091 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05e97f4-739f-48e4-a905-76db17a04058 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:11.815795 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46e6252-24a9-4b9e-bd6b-0bac79f6dff5 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:11.822882 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 266/524] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:03:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:11.823609 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05e97f4-739f-48e4-a905-76db17a04058 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:11.854787 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69243dc3-c778-4912-a4a0-3428cae7c438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:11.855443 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69243dc3-c778-4912-a4a0-3428cae7c438 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:11.855735 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69243dc3-c778-4912-a4a0-3428cae7c438 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:11.856011 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69243dc3-c778-4912-a4a0-3428cae7c438 None admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:11.856768 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69243dc3-c778-4912-a4a0-3428cae7c438 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:11.857076 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69243dc3-c778-4912-a4a0-3428cae7c438 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:11.858528 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69243dc3-c778-4912-a4a0-3428cae7c438 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:11.863869 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:11.881638 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 259/525] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Aug 4 21:03:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:11.881638 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69243dc3-c778-4912-a4a0-3428cae7c438 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:11.947410 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05e97f4-739f-48e4-a905-76db17a04058 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:11.982317 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:11.985455 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c61b930-946f-4d8e-8ce3-8f0c75ae0a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:11.986103 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c61b930-946f-4d8e-8ce3-8f0c75ae0a60 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:11.986382 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c61b930-946f-4d8e-8ce3-8f0c75ae0a60 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:11.987345 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c61b930-946f-4d8e-8ce3-8f0c75ae0a60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:11.990134 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69243dc3-c778-4912-a4a0-3428cae7c438 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:11.997816 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c61b930-946f-4d8e-8ce3-8f0c75ae0a60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:11.999350 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c61b930-946f-4d8e-8ce3-8f0c75ae0a60 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:12.015186 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-5c61b930-946f-4d8e-8ce3-8f0c75ae0a60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:12.015186 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:12.015186 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:12.015186 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:12.015186 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:12.015186 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:12.015186 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:12.015186 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:12.015186 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:12.015186 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:12.015186 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:12.015186 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:12.015186 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:03:12.015186 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:03:12.015186 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:03:12.016113 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:03:12.016113 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:12.016113 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:12.016113 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:12.016113 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:12.016113 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:12.016113 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:12.016113 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:12.016113 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:12.016113 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:12.016113 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:12.016113 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:12.016113 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:12.016113 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:12.016113 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:12.016113 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:12.016113 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:12.017042 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:03:12.017042 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:03:12.017042 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:12.017042 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:12.017042 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 267/526] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:03:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:12.017312 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c61b930-946f-4d8e-8ce3-8f0c75ae0a60 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:12.031680 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ed1c1d-3d3e-4881-bfc0-8fe972a7f7f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:12.032296 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ed1c1d-3d3e-4881-bfc0-8fe972a7f7f9 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:12.032577 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ed1c1d-3d3e-4881-bfc0-8fe972a7f7f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:12.032843 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ed1c1d-3d3e-4881-bfc0-8fe972a7f7f9 None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:12.034179 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ed1c1d-3d3e-4881-bfc0-8fe972a7f7f9 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:12.034501 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ed1c1d-3d3e-4881-bfc0-8fe972a7f7f9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:12.035979 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ed1c1d-3d3e-4881-bfc0-8fe972a7f7f9 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:12.054392 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 260/527] 10.222.0.25 () {56 vars in 1139 bytes} [Thu Aug 4 21:03:11 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:12.059015 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ed1c1d-3d3e-4881-bfc0-8fe972a7f7f9 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:12.116513 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:12.129622 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c61b930-946f-4d8e-8ce3-8f0c75ae0a60 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:12.161013 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ed1c1d-3d3e-4881-bfc0-8fe972a7f7f9 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:12.164424 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3517651e-b39b-49a9-b1ff-c3f99743a1da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:12.165055 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3517651e-b39b-49a9-b1ff-c3f99743a1da None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:12.165330 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3517651e-b39b-49a9-b1ff-c3f99743a1da None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:12.165596 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3517651e-b39b-49a9-b1ff-c3f99743a1da None admin] PATH_INFO: `/v3/projects/service` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:12.172474 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3517651e-b39b-49a9-b1ff-c3f99743a1da None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:12.173912 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3517651e-b39b-49a9-b1ff-c3f99743a1da None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:12.186211 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-3517651e-b39b-49a9-b1ff-c3f99743a1da None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:03:12.186211 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:12.186211 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:12.186211 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:12.186211 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:12.186211 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:12.186211 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:12.186211 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:12.186211 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:12.186211 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:12.186211 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:12.186211 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:12.186211 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 21:03:12.186211 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 21:03:12.186211 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 21:03:12.187061 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 21:03:12.187061 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:12.187061 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:12.187061 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:12.187061 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:12.187061 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:12.187061 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:12.187061 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:12.187061 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:12.187061 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:12.187061 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:12.187061 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:12.187061 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:12.187061 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:12.187061 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:12.187061 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:12.187061 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:12.187994 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 21:03:12.187994 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 21:03:12.187994 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 21:03:12.187994 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 21:03:12.187994 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 21:03:12.187994 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 21:03:12.187994 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:03:12.187994 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:12.192122 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 268/528] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Aug 4 21:03:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:12.192849 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3517651e-b39b-49a9-b1ff-c3f99743a1da None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:12.197499 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335e70cc-c945-4bd7-95f0-165e2de993eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:12.198116 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335e70cc-c945-4bd7-95f0-165e2de993eb None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:12.198392 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335e70cc-c945-4bd7-95f0-165e2de993eb None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:12.198664 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335e70cc-c945-4bd7-95f0-165e2de993eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:12.199483 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335e70cc-c945-4bd7-95f0-165e2de993eb None admin] RBAC: Adding query filter params (role.id=4eb30338d61e47808b51e050836643ea, scope.project.id=17dc3d39e2c048ca9070588f5717a578, user.id=6a2ce1df87fa47e5be301916bf3245fb) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:12.199792 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335e70cc-c945-4bd7-95f0-165e2de993eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:12.201704 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335e70cc-c945-4bd7-95f0-165e2de993eb None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:12.211345 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 261/529] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Aug 4 21:03:12 2022] GET /identity/v3/role_assignments?user.id=6a2ce1df87fa47e5be301916bf3245fb&scope.project.id=17dc3d39e2c048ca9070588f5717a578&role.id=4eb30338d61e47808b51e050836643ea => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:12.211345 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335e70cc-c945-4bd7-95f0-165e2de993eb None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:12.241463 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:12.332533 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3517651e-b39b-49a9-b1ff-c3f99743a1da None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:12.354192 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335e70cc-c945-4bd7-95f0-165e2de993eb None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:12.375448 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9f10c0-8a70-4621-9942-73eb7fd7f15b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:12.377419 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9f10c0-8a70-4621-9942-73eb7fd7f15b None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:12.377821 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9f10c0-8a70-4621-9942-73eb7fd7f15b None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:12.380357 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9f10c0-8a70-4621-9942-73eb7fd7f15b None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:12.380357 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9f10c0-8a70-4621-9942-73eb7fd7f15b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:12.380357 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9f10c0-8a70-4621-9942-73eb7fd7f15b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:12.380711 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9f10c0-8a70-4621-9942-73eb7fd7f15b None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:12.395864 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 269/530] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:03:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:12.399689 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9f10c0-8a70-4621-9942-73eb7fd7f15b None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:12.401478 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb0cddd-5485-484a-a4a1-f3b85b0d16a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:12.402106 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb0cddd-5485-484a-a4a1-f3b85b0d16a5 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:12.402386 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb0cddd-5485-484a-a4a1-f3b85b0d16a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:12.403074 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb0cddd-5485-484a-a4a1-f3b85b0d16a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:12.414478 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:12.497689 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb0cddd-5485-484a-a4a1-f3b85b0d16a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:12.499626 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb0cddd-5485-484a-a4a1-f3b85b0d16a5 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:12.521688 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:12.523910 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9f10c0-8a70-4621-9942-73eb7fd7f15b None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:12.570917 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b1ab74-d084-46ae-8dc7-d375f4db67c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:12.571574 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b1ab74-d084-46ae-8dc7-d375f4db67c3 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:12.571857 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b1ab74-d084-46ae-8dc7-d375f4db67c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:12.572152 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b1ab74-d084-46ae-8dc7-d375f4db67c3 None admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:12.573143 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b1ab74-d084-46ae-8dc7-d375f4db67c3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:12.573453 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b1ab74-d084-46ae-8dc7-d375f4db67c3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:12.574896 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b1ab74-d084-46ae-8dc7-d375f4db67c3 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:12.603406 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 270/531] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Aug 4 21:03:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:12.604606 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b1ab74-d084-46ae-8dc7-d375f4db67c3 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:12.622686 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 262/532] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Aug 4 21:03:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 21:03:12.624003 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb0cddd-5485-484a-a4a1-f3b85b0d16a5 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:12.686544 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:12.721828 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b1ab74-d084-46ae-8dc7-d375f4db67c3 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:12.755117 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb0cddd-5485-484a-a4a1-f3b85b0d16a5 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:12.763210 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80577e4-2914-4151-a4d9-ed4388c721e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:12.764393 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80577e4-2914-4151-a4d9-ed4388c721e9 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:12.764679 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80577e4-2914-4151-a4d9-ed4388c721e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:12.764953 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80577e4-2914-4151-a4d9-ed4388c721e9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:12.774320 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:12.775196 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80577e4-2914-4151-a4d9-ed4388c721e9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:12.776813 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80577e4-2914-4151-a4d9-ed4388c721e9 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:12.791960 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-a80577e4-2914-4151-a4d9-ed4388c721e9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:03:12.791960 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:12.791960 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:12.791960 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:12.791960 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:12.791960 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:12.791960 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:12.791960 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:12.791960 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:12.791960 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:12.791960 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:12.791960 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:12.791960 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 21:03:12.791960 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 21:03:12.791960 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 21:03:12.792866 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 21:03:12.792866 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:12.792866 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:12.792866 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:12.792866 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:12.792866 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:12.792866 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:12.792866 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:12.792866 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:12.792866 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:12.792866 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:12.792866 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:12.792866 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:12.792866 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:12.792866 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:12.792866 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:12.792866 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:12.801803 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 21:03:12.801803 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 21:03:12.801803 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 21:03:12.801803 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 21:03:12.801803 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 21:03:12.801803 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 21:03:12.801803 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:03:12.801803 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:12.801803 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 271/533] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Aug 4 21:03:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:12.801803 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80577e4-2914-4151-a4d9-ed4388c721e9 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:12.809704 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f91cacd-49b5-4478-b283-f82e7e766a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:12.810645 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f91cacd-49b5-4478-b283-f82e7e766a40 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:12.811193 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f91cacd-49b5-4478-b283-f82e7e766a40 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:12.811710 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f91cacd-49b5-4478-b283-f82e7e766a40 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:12.812852 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f91cacd-49b5-4478-b283-f82e7e766a40 None admin] RBAC: Adding query filter params (role.id=4eb30338d61e47808b51e050836643ea, scope.project.id=17dc3d39e2c048ca9070588f5717a578, user.id=e31f2728056b460fa474478eb85ca1d8) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:12.813413 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f91cacd-49b5-4478-b283-f82e7e766a40 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:12.817704 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f91cacd-49b5-4478-b283-f82e7e766a40 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:12.824762 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 263/534] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Aug 4 21:03:12 2022] GET /identity/v3/role_assignments?user.id=e31f2728056b460fa474478eb85ca1d8&scope.project.id=17dc3d39e2c048ca9070588f5717a578&role.id=4eb30338d61e47808b51e050836643ea => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:12.825961 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f91cacd-49b5-4478-b283-f82e7e766a40 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:12.934026 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80577e4-2914-4151-a4d9-ed4388c721e9 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:12.963861 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:12.976741 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c629109-e8e7-4568-835a-c4a73d09d268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:12.978550 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c629109-e8e7-4568-835a-c4a73d09d268 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:12.980001 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c629109-e8e7-4568-835a-c4a73d09d268 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:12.980363 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c629109-e8e7-4568-835a-c4a73d09d268 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:12.985909 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f91cacd-49b5-4478-b283-f82e7e766a40 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:12.996437 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c629109-e8e7-4568-835a-c4a73d09d268 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:12.998650 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c629109-e8e7-4568-835a-c4a73d09d268 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:13.022125 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-0c629109-e8e7-4568-835a-c4a73d09d268 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:13.022125 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:13.022125 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:13.022125 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:13.022125 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:13.022125 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:13.022125 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:13.022125 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:13.022125 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:13.022125 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:13.022125 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:13.022125 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:13.022125 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:03:13.022125 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:03:13.022125 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:03:13.023030 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:03:13.023030 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:13.023030 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:13.023030 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:13.023030 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:13.023030 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:13.023030 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:13.023030 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:13.023030 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:13.023030 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:13.023030 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:13.023030 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:13.023030 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:13.023030 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:13.023030 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:13.023030 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:13.023030 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:13.024303 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:03:13.024303 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:03:13.024303 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:13.024303 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:13.031432 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 272/535] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:03:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:13.031432 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1688f26b-583f-44a3-97b2-d8826d87074e None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:13.031432 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1688f26b-583f-44a3-97b2-d8826d87074e None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:13.031432 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1688f26b-583f-44a3-97b2-d8826d87074e None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:13.031817 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 273/536] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:03:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:03:13.032522 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1688f26b-583f-44a3-97b2-d8826d87074e None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:13.053526 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5158478-86c4-4252-ae72-9b6e6a61be31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:13.054459 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5158478-86c4-4252-ae72-9b6e6a61be31 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:13.055193 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5158478-86c4-4252-ae72-9b6e6a61be31 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:13.055580 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5158478-86c4-4252-ae72-9b6e6a61be31 None admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:13.056419 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5158478-86c4-4252-ae72-9b6e6a61be31 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:13.056804 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5158478-86c4-4252-ae72-9b6e6a61be31 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:13.058898 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5158478-86c4-4252-ae72-9b6e6a61be31 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:13.076064 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 264/537] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Aug 4 21:03:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:13.077384 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac0df60-c110-42cc-87d4-f2a4cf057ddc None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:13.077752 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac0df60-c110-42cc-87d4-f2a4cf057ddc None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:13.078086 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac0df60-c110-42cc-87d4-f2a4cf057ddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:13.100214 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-6ac0df60-c110-42cc-87d4-f2a4cf057ddc None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:03:13.101579 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-6ac0df60-c110-42cc-87d4-f2a4cf057ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:03:13.138046 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:13.160589 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1688f26b-583f-44a3-97b2-d8826d87074e None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:13.182540 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 265/538] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:03:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:03:13.184179 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac0df60-c110-42cc-87d4-f2a4cf057ddc None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:13.217282 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb40172-621a-437e-b122-731679253988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:13.217934 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb40172-621a-437e-b122-731679253988 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:13.218228 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb40172-621a-437e-b122-731679253988 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:13.218567 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb40172-621a-437e-b122-731679253988 None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:13.219801 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb40172-621a-437e-b122-731679253988 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:13.219801 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb40172-621a-437e-b122-731679253988 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:13.222162 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb40172-621a-437e-b122-731679253988 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:13.247239 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 274/539] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:03:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:13.248295 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb40172-621a-437e-b122-731679253988 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:13.295689 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:13.299642 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac0df60-c110-42cc-87d4-f2a4cf057ddc None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:13.336343 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d4e635-e7ea-497f-ba12-d153e6abf2d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:13.337124 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d4e635-e7ea-497f-ba12-d153e6abf2d3 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:13.337484 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d4e635-e7ea-497f-ba12-d153e6abf2d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:13.337769 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d4e635-e7ea-497f-ba12-d153e6abf2d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:13.338599 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d4e635-e7ea-497f-ba12-d153e6abf2d3 None admin] RBAC: Adding query filter params (role.id=4eb30338d61e47808b51e050836643ea, scope.project.id=17dc3d39e2c048ca9070588f5717a578, user.id=f6885c7d929f4f8e96d3890724c3b53f) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:13.338914 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d4e635-e7ea-497f-ba12-d153e6abf2d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:13.340579 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d4e635-e7ea-497f-ba12-d153e6abf2d3 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:13.355481 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 266/540] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Aug 4 21:03:13 2022] GET /identity/v3/role_assignments?user.id=f6885c7d929f4f8e96d3890724c3b53f&scope.project.id=17dc3d39e2c048ca9070588f5717a578&role.id=4eb30338d61e47808b51e050836643ea => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:13.356703 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d4e635-e7ea-497f-ba12-d153e6abf2d3 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:13.362102 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb40172-621a-437e-b122-731679253988 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:13.402526 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be45ac2e-be06-4445-9400-e909ddb122f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:13.403218 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be45ac2e-be06-4445-9400-e909ddb122f9 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:13.403756 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be45ac2e-be06-4445-9400-e909ddb122f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:13.404036 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be45ac2e-be06-4445-9400-e909ddb122f9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:13.426927 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:13.429737 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be45ac2e-be06-4445-9400-e909ddb122f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:03:13.441692 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-be45ac2e-be06-4445-9400-e909ddb122f9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:03:13.441692 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:13.441692 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:13.441692 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:13.441692 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:13.441692 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:13.441692 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:13.441692 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:13.441692 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:13.441692 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:13.441692 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:13.441692 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:13.441692 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:03:13.441692 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:03:13.441692 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:03:13.441692 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:03:13.442652 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:03:13.442652 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:03:13.442652 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:03:13.442652 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:03:13.442652 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:13.442652 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:13.442652 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:13.442652 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:13.442652 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:13.442652 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:13.442652 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:13.442652 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:13.442652 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:13.442652 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:13.442652 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:13.442652 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:13.443914 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:13.443914 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:13.443914 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:13.443914 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:13.443914 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:03:13.443914 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:03:13.443914 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:03:13.443914 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:03:13.443914 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:03:13.443914 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:03:13.443914 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:03:13.443914 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:13.443914 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 275/541] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Aug 4 21:03:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:13.451854 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be45ac2e-be06-4445-9400-e909ddb122f9 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:13.560322 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d4e635-e7ea-497f-ba12-d153e6abf2d3 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:13.604625 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:13.608963 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be45ac2e-be06-4445-9400-e909ddb122f9 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:13.613435 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f5c406-d7cd-4f03-b4cf-778a5b879763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:13.614099 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f5c406-d7cd-4f03-b4cf-778a5b879763 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:13.614385 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f5c406-d7cd-4f03-b4cf-778a5b879763 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:13.614659 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f5c406-d7cd-4f03-b4cf-778a5b879763 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:13.626136 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f5c406-d7cd-4f03-b4cf-778a5b879763 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:13.628281 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f5c406-d7cd-4f03-b4cf-778a5b879763 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:13.651651 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-25f5c406-d7cd-4f03-b4cf-778a5b879763 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:03:13.651651 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:13.651651 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:13.651651 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:13.651651 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:13.651651 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:13.651651 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:13.651651 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:13.651651 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:13.651651 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:13.651651 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:13.651651 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:13.651651 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 21:03:13.651651 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 21:03:13.651651 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 21:03:13.652543 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 21:03:13.652543 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:13.652543 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:13.652543 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:13.652543 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:13.652543 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:13.652543 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:13.652543 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:13.652543 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:13.652543 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:13.652543 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:13.652543 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:13.652543 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:13.652543 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:13.652543 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:13.652543 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:13.652543 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:13.653474 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 21:03:13.653474 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 21:03:13.653474 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 21:03:13.653474 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 21:03:13.653474 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 21:03:13.653474 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 21:03:13.653474 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:03:13.653474 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:13.653879 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0544438-d306-4652-93d5-9e22f67e232d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:13.654443 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0544438-d306-4652-93d5-9e22f67e232d None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:13.654443 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0544438-d306-4652-93d5-9e22f67e232d None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:13.654676 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0544438-d306-4652-93d5-9e22f67e232d None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:13.659349 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0544438-d306-4652-93d5-9e22f67e232d None admin] RBAC: Adding query filter params (name=service) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:13.659349 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0544438-d306-4652-93d5-9e22f67e232d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:13.659349 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0544438-d306-4652-93d5-9e22f67e232d None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:13.659575 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 267/542] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Aug 4 21:03:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:13.659575 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f5c406-d7cd-4f03-b4cf-778a5b879763 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:13.675658 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 276/543] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Aug 4 21:03:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:13.680865 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0544438-d306-4652-93d5-9e22f67e232d None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:13.779343 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f5c406-d7cd-4f03-b4cf-778a5b879763 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:13.781802 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:13.835887 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf949ea-a93a-43ba-84a2-f0cf0b3a76bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:13.836436 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf949ea-a93a-43ba-84a2-f0cf0b3a76bd None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:13.836728 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf949ea-a93a-43ba-84a2-f0cf0b3a76bd None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:13.837012 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf949ea-a93a-43ba-84a2-f0cf0b3a76bd None admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:13.837774 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf949ea-a93a-43ba-84a2-f0cf0b3a76bd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:13.838088 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf949ea-a93a-43ba-84a2-f0cf0b3a76bd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:13.839674 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf949ea-a93a-43ba-84a2-f0cf0b3a76bd None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:13.855366 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 268/544] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Aug 4 21:03:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:03:13.857982 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf949ea-a93a-43ba-84a2-f0cf0b3a76bd None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:13.904488 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0544438-d306-4652-93d5-9e22f67e232d None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:13.977241 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90712a10-2bd0-447c-afa1-7c7e37ed3072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:13.978340 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90712a10-2bd0-447c-afa1-7c7e37ed3072 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:13.978633 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90712a10-2bd0-447c-afa1-7c7e37ed3072 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:13.978913 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90712a10-2bd0-447c-afa1-7c7e37ed3072 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:13.991625 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90712a10-2bd0-447c-afa1-7c7e37ed3072 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:13.993460 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90712a10-2bd0-447c-afa1-7c7e37ed3072 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:14.006468 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf949ea-a93a-43ba-84a2-f0cf0b3a76bd None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:14.011376 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-90712a10-2bd0-447c-afa1-7c7e37ed3072 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:14.011376 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:14.011376 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:14.011376 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:14.011376 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:14.011376 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:14.011376 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:14.011376 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:14.011376 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:14.011376 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:14.011376 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:14.011376 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:14.011376 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:03:14.011376 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:03:14.011376 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:03:14.015459 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:03:14.015459 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:14.015459 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:14.015459 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:14.015459 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:14.015459 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:14.015459 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:14.015459 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:14.015459 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:14.015459 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:14.015459 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:14.015459 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:14.015459 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:14.015459 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:14.015459 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:14.015459 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:14.015459 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:14.016310 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:03:14.016310 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:03:14.016310 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:14.016310 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:14.016310 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 277/545] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:03:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:03:14.019270 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90712a10-2bd0-447c-afa1-7c7e37ed3072 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:14.055660 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:14.090911 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7147d7f1-0c73-4185-a0eb-07892bb558fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:14.091480 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7147d7f1-0c73-4185-a0eb-07892bb558fc None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:14.091768 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7147d7f1-0c73-4185-a0eb-07892bb558fc None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:14.092046 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7147d7f1-0c73-4185-a0eb-07892bb558fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:14.095887 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7147d7f1-0c73-4185-a0eb-07892bb558fc None admin] RBAC: Adding query filter params (role.id=4eb30338d61e47808b51e050836643ea, scope.project.id=17dc3d39e2c048ca9070588f5717a578, user.id=08fcb91ee54344b5bb3870c585e25fdc) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:14.096215 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7147d7f1-0c73-4185-a0eb-07892bb558fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:14.097801 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7147d7f1-0c73-4185-a0eb-07892bb558fc None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:14.108817 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 269/546] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Aug 4 21:03:13 2022] GET /identity/v3/role_assignments?user.id=08fcb91ee54344b5bb3870c585e25fdc&scope.project.id=17dc3d39e2c048ca9070588f5717a578&role.id=4eb30338d61e47808b51e050836643ea => generated 259 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:03:14.204055 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90712a10-2bd0-447c-afa1-7c7e37ed3072 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:14.265542 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d262991-be25-42f4-aaa4-f394cc9c14d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:14.266200 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d262991-be25-42f4-aaa4-f394cc9c14d2 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:14.266485 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d262991-be25-42f4-aaa4-f394cc9c14d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:14.266763 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d262991-be25-42f4-aaa4-f394cc9c14d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:14.267507 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d262991-be25-42f4-aaa4-f394cc9c14d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:14.267880 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d262991-be25-42f4-aaa4-f394cc9c14d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:14.269778 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d262991-be25-42f4-aaa4-f394cc9c14d2 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:14.290014 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 278/547] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:03:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:14.295193 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7147d7f1-0c73-4185-a0eb-07892bb558fc None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:14.319745 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:14.348316 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274737b8-9da6-4571-9b41-bcf5282cfe39 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:14.348696 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274737b8-9da6-4571-9b41-bcf5282cfe39 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:14.348933 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274737b8-9da6-4571-9b41-bcf5282cfe39 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:14.350751 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 279/548] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:03:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 21:03:14.356139 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d920862-73a1-4e84-bf9a-f9e49ebde050 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:14.356320 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d920862-73a1-4e84-bf9a-f9e49ebde050 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:14.356799 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d920862-73a1-4e84-bf9a-f9e49ebde050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:14.375268 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-0d920862-73a1-4e84-bf9a-f9e49ebde050 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:03:14.376583 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-0d920862-73a1-4e84-bf9a-f9e49ebde050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:03:14.458237 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 280/549] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:03:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:03:14.481845 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d920862-73a1-4e84-bf9a-f9e49ebde050 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:14.485387 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:14.492288 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7147d7f1-0c73-4185-a0eb-07892bb558fc None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:14.592247 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae849e45-2dd5-4119-89c9-75d2c9df787a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:14.592901 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae849e45-2dd5-4119-89c9-75d2c9df787a None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:14.593194 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae849e45-2dd5-4119-89c9-75d2c9df787a None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:14.593475 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae849e45-2dd5-4119-89c9-75d2c9df787a None admin] PATH_INFO: `/v3/users/swift` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:14.598740 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d920862-73a1-4e84-bf9a-f9e49ebde050 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:14.616164 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae849e45-2dd5-4119-89c9-75d2c9df787a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:14.617661 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae849e45-2dd5-4119-89c9-75d2c9df787a None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:14.640035 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eff3f0-5ea2-4845-ab8d-b6c3b852fd90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:14.640848 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eff3f0-5ea2-4845-ab8d-b6c3b852fd90 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:14.641140 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eff3f0-5ea2-4845-ab8d-b6c3b852fd90 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:14.641248 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-ae849e45-2dd5-4119-89c9-75d2c9df787a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 04 21:03:14.641248 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:14.641248 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:14.641248 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:14.641248 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:14.641248 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:14.641248 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:14.641248 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:14.641248 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:14.641248 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:14.641248 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:14.641248 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:14.641248 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 21:03:14.641248 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 21:03:14.641248 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 21:03:14.641248 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 21:03:14.642049 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:14.642049 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:14.642049 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 21:03:14.642049 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:03:14.642049 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 21:03:14.642049 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:03:14.642049 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:14.642049 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:14.642049 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:14.642049 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:14.642049 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:14.642049 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:14.642049 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:14.642049 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:14.642049 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:14.642049 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:14.642049 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:14.642049 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:14.643478 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:14.643478 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:14.643478 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 21:03:14.643478 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 21:03:14.643478 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 21:03:14.643478 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 21:03:14.643478 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 21:03:14.643478 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 21:03:14.643478 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 04 21:03:14.643478 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:14.644010 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eff3f0-5ea2-4845-ab8d-b6c3b852fd90 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:14.644079 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 270/550] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Aug 4 21:03:14 2022] GET /identity/v3/users/swift => generated 83 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:14.649773 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae849e45-2dd5-4119-89c9-75d2c9df787a None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:14.658957 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eff3f0-5ea2-4845-ab8d-b6c3b852fd90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:03:14.665251 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-f2eff3f0-5ea2-4845-ab8d-b6c3b852fd90 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:03:14.665251 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:14.665251 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:14.665251 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:14.665251 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:14.665251 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:14.665251 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:14.665251 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:14.665251 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:14.665251 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:14.665251 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:14.665251 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:14.665251 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:03:14.665251 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:03:14.665251 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:03:14.665251 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:03:14.667599 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:03:14.667599 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:03:14.667599 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:03:14.667599 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:03:14.667599 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:14.667599 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:14.667599 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:14.667599 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:14.667599 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:14.667599 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:14.667599 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:14.667599 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:14.667599 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:14.667599 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:14.667599 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:14.667599 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:14.669540 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:14.669540 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:14.669540 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:14.669540 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:14.669540 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:03:14.669540 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:03:14.669540 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:03:14.669540 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:03:14.669540 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:03:14.669540 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:03:14.669540 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:03:14.669540 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:14.669540 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 281/551] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Aug 4 21:03:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:14.675349 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eff3f0-5ea2-4845-ab8d-b6c3b852fd90 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:14.793888 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:14.806320 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae849e45-2dd5-4119-89c9-75d2c9df787a None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:14.834230 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:14.847109 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35675da1-d623-44d2-8473-3896d57076b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:14.847816 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35675da1-d623-44d2-8473-3896d57076b5 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:14.848103 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35675da1-d623-44d2-8473-3896d57076b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:14.848382 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35675da1-d623-44d2-8473-3896d57076b5 None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:14.849246 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35675da1-d623-44d2-8473-3896d57076b5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:14.849505 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35675da1-d623-44d2-8473-3896d57076b5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:14.850967 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35675da1-d623-44d2-8473-3896d57076b5 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:14.876873 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 271/552] 10.222.0.25 () {56 vars in 1135 bytes} [Thu Aug 4 21:03:14 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:14.879588 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35675da1-d623-44d2-8473-3896d57076b5 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:14.995008 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35675da1-d623-44d2-8473-3896d57076b5 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:15.017397 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eff3f0-5ea2-4845-ab8d-b6c3b852fd90 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:15.034186 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9707d887-a236-4f1b-baf4-f1589bb11f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:15.034820 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9707d887-a236-4f1b-baf4-f1589bb11f5e None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:15.035106 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9707d887-a236-4f1b-baf4-f1589bb11f5e None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:15.035435 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9707d887-a236-4f1b-baf4-f1589bb11f5e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:15.039223 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:15.119634 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9707d887-a236-4f1b-baf4-f1589bb11f5e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:15.121086 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9707d887-a236-4f1b-baf4-f1589bb11f5e None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:15.192560 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b4a175-6b27-4890-9073-bc2aee9466cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:15.193195 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b4a175-6b27-4890-9073-bc2aee9466cc None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:15.193477 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b4a175-6b27-4890-9073-bc2aee9466cc None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:15.193753 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b4a175-6b27-4890-9073-bc2aee9466cc None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:15.194475 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b4a175-6b27-4890-9073-bc2aee9466cc None admin] RBAC: Adding query filter params (name=service) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:15.194777 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b4a175-6b27-4890-9073-bc2aee9466cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:15.196199 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b4a175-6b27-4890-9073-bc2aee9466cc None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:15.225937 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 282/553] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Aug 4 21:03:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 553 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:03:15.240056 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 272/554] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Aug 4 21:03:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 04 21:03:15.240225 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b4a175-6b27-4890-9073-bc2aee9466cc None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:15.241331 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9707d887-a236-4f1b-baf4-f1589bb11f5e None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:15.299685 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:15.350735 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9707d887-a236-4f1b-baf4-f1589bb11f5e None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:15.387970 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bf049f-02a0-4ed8-8935-23cd1d9a497f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:15.388596 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bf049f-02a0-4ed8-8935-23cd1d9a497f None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:15.388886 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bf049f-02a0-4ed8-8935-23cd1d9a497f None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:15.389162 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bf049f-02a0-4ed8-8935-23cd1d9a497f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:15.403060 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bf049f-02a0-4ed8-8935-23cd1d9a497f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:15.405599 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bf049f-02a0-4ed8-8935-23cd1d9a497f None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:15.426812 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-f5bf049f-02a0-4ed8-8935-23cd1d9a497f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:15.426812 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:15.426812 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:15.426812 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:15.426812 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:15.426812 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:15.426812 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:15.426812 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:15.426812 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:15.426812 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:15.426812 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:15.426812 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:15.426812 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:03:15.426812 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:03:15.426812 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:03:15.427807 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:03:15.427807 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:15.427807 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:15.427807 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:15.427807 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:15.427807 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:15.427807 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:15.427807 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:15.427807 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:15.427807 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:15.427807 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:15.427807 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:15.427807 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:15.427807 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:15.427807 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:15.427807 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:15.427807 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:15.429119 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:03:15.429119 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:03:15.429119 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:15.429119 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:15.431289 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b4a175-6b27-4890-9073-bc2aee9466cc None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:15.432710 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 273/555] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:03:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:15.434256 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25ed95c-a223-46d1-afd3-23a916872875 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:15.434556 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25ed95c-a223-46d1-afd3-23a916872875 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:15.435687 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25ed95c-a223-46d1-afd3-23a916872875 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:15.440511 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 274/556] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:03:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:03:15.441983 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25ed95c-a223-46d1-afd3-23a916872875 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:15.467333 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:15.477463 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ff429b-8081-479e-836e-9c5335718d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:15.478363 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ff429b-8081-479e-836e-9c5335718d03 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:15.478796 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ff429b-8081-479e-836e-9c5335718d03 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:15.479342 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ff429b-8081-479e-836e-9c5335718d03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:15.491623 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ff429b-8081-479e-836e-9c5335718d03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:15.493428 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ff429b-8081-479e-836e-9c5335718d03 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:15.510932 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-45ff429b-8081-479e-836e-9c5335718d03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:15.510932 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:15.510932 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:15.510932 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:15.510932 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:15.510932 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:15.510932 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:15.510932 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:15.510932 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:15.510932 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:15.510932 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:15.510932 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:15.510932 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:03:15.510932 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:03:15.510932 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:03:15.512023 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:03:15.512023 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:15.512023 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:15.512023 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:15.512023 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:15.512023 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:15.512023 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:15.512023 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:15.512023 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:15.512023 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:15.512023 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:15.512023 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:15.512023 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:15.512023 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:15.512023 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:15.512023 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:15.512023 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:15.513588 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:03:15.513588 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:03:15.513588 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:15.513588 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:15.515505 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 283/557] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:03:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:15.516892 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7171c095-2acb-42f4-be91-43170684a2fc None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:15.517499 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7171c095-2acb-42f4-be91-43170684a2fc None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:15.518040 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7171c095-2acb-42f4-be91-43170684a2fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:15.537751 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-7171c095-2acb-42f4-be91-43170684a2fc None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:03:15.539473 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-7171c095-2acb-42f4-be91-43170684a2fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:03:15.541464 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:15.555200 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25ed95c-a223-46d1-afd3-23a916872875 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:15.591031 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d50cd61-ee7e-4466-a63e-17cee98189ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:15.591720 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d50cd61-ee7e-4466-a63e-17cee98189ff None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:15.592007 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d50cd61-ee7e-4466-a63e-17cee98189ff None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:15.592335 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d50cd61-ee7e-4466-a63e-17cee98189ff None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:15.593035 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d50cd61-ee7e-4466-a63e-17cee98189ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:15.593401 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d50cd61-ee7e-4466-a63e-17cee98189ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:15.594848 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d50cd61-ee7e-4466-a63e-17cee98189ff None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:15.607194 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 275/558] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:03:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:15.608255 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d50cd61-ee7e-4466-a63e-17cee98189ff None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:15.623444 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 284/559] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:03:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:03:15.624707 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7171c095-2acb-42f4-be91-43170684a2fc None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:15.713628 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d50cd61-ee7e-4466-a63e-17cee98189ff None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:15.748670 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a022169-2159-4799-8b39-2c3b2c83059a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:15.749325 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a022169-2159-4799-8b39-2c3b2c83059a None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:15.749576 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a022169-2159-4799-8b39-2c3b2c83059a None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:15.749857 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a022169-2159-4799-8b39-2c3b2c83059a None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:15.750555 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a022169-2159-4799-8b39-2c3b2c83059a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:15.750888 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a022169-2159-4799-8b39-2c3b2c83059a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:15.752429 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a022169-2159-4799-8b39-2c3b2c83059a None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:15.754616 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7171c095-2acb-42f4-be91-43170684a2fc None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:15.764907 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:15.765258 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 276/560] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:03:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:15.766384 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a022169-2159-4799-8b39-2c3b2c83059a None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:15.777167 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:15.794800 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160881e8-8cc4-4556-bca5-57befd88d8bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:15.795779 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160881e8-8cc4-4556-bca5-57befd88d8bc None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:15.796248 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160881e8-8cc4-4556-bca5-57befd88d8bc None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:15.796719 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160881e8-8cc4-4556-bca5-57befd88d8bc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:15.804597 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160881e8-8cc4-4556-bca5-57befd88d8bc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:15.806300 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160881e8-8cc4-4556-bca5-57befd88d8bc None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:15.818603 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-160881e8-8cc4-4556-bca5-57befd88d8bc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:03:15.818603 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:15.818603 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:15.818603 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:15.818603 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:15.818603 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:15.818603 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:15.818603 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:15.818603 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:15.818603 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:15.818603 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:15.818603 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:15.818603 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 21:03:15.818603 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 21:03:15.818603 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 21:03:15.819586 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 21:03:15.819586 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:15.819586 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:15.819586 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:15.819586 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:15.819586 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:15.819586 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:15.819586 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:15.819586 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:15.819586 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:15.819586 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:15.819586 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:15.819586 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:15.819586 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:15.819586 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:15.819586 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:15.819586 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:15.820486 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 21:03:15.820486 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 21:03:15.820486 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 21:03:15.820486 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 21:03:15.820486 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 21:03:15.820486 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 21:03:15.820486 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:03:15.820486 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:15.822889 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 285/561] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Aug 4 21:03:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:15.824033 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160881e8-8cc4-4556-bca5-57befd88d8bc None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:15.872118 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a022169-2159-4799-8b39-2c3b2c83059a None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:15.909472 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1309e3b-7837-448e-92fb-f725100f0146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:15.910178 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1309e3b-7837-448e-92fb-f725100f0146 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:15.910462 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1309e3b-7837-448e-92fb-f725100f0146 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:15.910787 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1309e3b-7837-448e-92fb-f725100f0146 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:15.924444 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1309e3b-7837-448e-92fb-f725100f0146 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:03:15.931551 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-e1309e3b-7837-448e-92fb-f725100f0146 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:03:15.931551 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:15.931551 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:15.931551 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:15.931551 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:15.931551 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:15.931551 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:15.931551 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:15.931551 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:15.931551 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:15.931551 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:15.931551 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:15.931551 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:03:15.931551 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:03:15.931551 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:03:15.931551 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:03:15.932628 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:03:15.932628 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:03:15.932628 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:03:15.932628 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:03:15.932628 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:15.932628 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:15.932628 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:15.932628 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:15.932628 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:15.932628 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:15.932628 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:15.932628 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:15.932628 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:15.932628 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:15.932628 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:15.932628 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:15.933480 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:15.933480 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:15.933480 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:15.933480 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:15.933480 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:03:15.933480 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:03:15.933480 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:03:15.933480 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:03:15.933480 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:03:15.933480 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:03:15.933480 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:03:15.933480 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:15.935405 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 277/562] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Aug 4 21:03:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:15.936506 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1309e3b-7837-448e-92fb-f725100f0146 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:15.950122 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160881e8-8cc4-4556-bca5-57befd88d8bc None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:15.994548 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ea1d35-7b67-4f2b-97ec-c1e026ff863c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:15.995355 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ea1d35-7b67-4f2b-97ec-c1e026ff863c None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:15.995755 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ea1d35-7b67-4f2b-97ec-c1e026ff863c None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:15.996142 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ea1d35-7b67-4f2b-97ec-c1e026ff863c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:16.004330 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ea1d35-7b67-4f2b-97ec-c1e026ff863c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:16.005896 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ea1d35-7b67-4f2b-97ec-c1e026ff863c None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:16.010466 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:16.019359 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-32ea1d35-7b67-4f2b-97ec-c1e026ff863c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 21:03:16.019359 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:16.019359 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:16.019359 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:16.019359 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:16.019359 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:16.019359 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:16.019359 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:16.019359 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:16.019359 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:16.019359 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:16.019359 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:16.019359 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 21:03:16.019359 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 21:03:16.019359 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 21:03:16.019359 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 21:03:16.020537 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:16.020537 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:16.020537 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 21:03:16.020537 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:03:16.020537 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 21:03:16.020537 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:03:16.020537 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:16.020537 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:16.020537 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:16.020537 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:16.020537 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:16.020537 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:16.020537 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:16.020537 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:16.020537 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:16.020537 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:16.020537 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:16.020537 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:16.022307 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:16.022307 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:16.022307 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 21:03:16.022307 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 21:03:16.022307 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 21:03:16.022307 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 21:03:16.022307 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 21:03:16.022307 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 21:03:16.022307 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 21:03:16.022307 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:16.025109 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 286/563] 10.222.0.25 () {56 vars in 1093 bytes} [Thu Aug 4 21:03:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:16.025786 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ea1d35-7b67-4f2b-97ec-c1e026ff863c None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:16.049839 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1309e3b-7837-448e-92fb-f725100f0146 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:16.056355 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:16.086782 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea83795-b563-4cf9-9ee6-4b0577937ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:16.087775 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea83795-b563-4cf9-9ee6-4b0577937ebd None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:16.088200 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea83795-b563-4cf9-9ee6-4b0577937ebd None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:16.088482 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea83795-b563-4cf9-9ee6-4b0577937ebd None admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:16.089239 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea83795-b563-4cf9-9ee6-4b0577937ebd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:16.089554 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea83795-b563-4cf9-9ee6-4b0577937ebd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:16.091022 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea83795-b563-4cf9-9ee6-4b0577937ebd None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:16.109871 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 278/564] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Aug 4 21:03:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:16.111941 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f910bf11-9dd5-47c6-aacc-2beb56f02bed None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:16.112381 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f910bf11-9dd5-47c6-aacc-2beb56f02bed None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:16.112667 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f910bf11-9dd5-47c6-aacc-2beb56f02bed None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:16.115245 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 279/565] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:03:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:03:16.115245 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f910bf11-9dd5-47c6-aacc-2beb56f02bed None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:16.138913 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ea1d35-7b67-4f2b-97ec-c1e026ff863c None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:16.176460 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0cbb8a-2d07-4c1c-bc7e-4cafc2a23c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:16.177138 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0cbb8a-2d07-4c1c-bc7e-4cafc2a23c31 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:16.177428 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0cbb8a-2d07-4c1c-bc7e-4cafc2a23c31 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:16.177926 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0cbb8a-2d07-4c1c-bc7e-4cafc2a23c31 None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:16.178416 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0cbb8a-2d07-4c1c-bc7e-4cafc2a23c31 None admin] RBAC: Adding query filter params (name=service) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:16.178721 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0cbb8a-2d07-4c1c-bc7e-4cafc2a23c31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:16.180147 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0cbb8a-2d07-4c1c-bc7e-4cafc2a23c31 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:16.188858 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 287/566] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Aug 4 21:03:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:16.189720 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0cbb8a-2d07-4c1c-bc7e-4cafc2a23c31 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:16.233888 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f910bf11-9dd5-47c6-aacc-2beb56f02bed None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:16.265390 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:16.274977 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc764bda-a8dc-4f37-8d4c-c8190aa1684a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:16.275629 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc764bda-a8dc-4f37-8d4c-c8190aa1684a None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:16.280308 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc764bda-a8dc-4f37-8d4c-c8190aa1684a None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:16.280308 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc764bda-a8dc-4f37-8d4c-c8190aa1684a None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:16.280308 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc764bda-a8dc-4f37-8d4c-c8190aa1684a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:16.280308 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc764bda-a8dc-4f37-8d4c-c8190aa1684a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:16.280308 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc764bda-a8dc-4f37-8d4c-c8190aa1684a None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:16.290785 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:16.297132 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 280/567] 10.222.0.25 () {56 vars in 1133 bytes} [Thu Aug 4 21:03:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:16.298439 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e1d733-cc38-4553-ad1c-25d9f2a3a630 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:16.298734 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e1d733-cc38-4553-ad1c-25d9f2a3a630 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:16.299569 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e1d733-cc38-4553-ad1c-25d9f2a3a630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:16.306215 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0cbb8a-2d07-4c1c-bc7e-4cafc2a23c31 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:16.317726 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-d9e1d733-cc38-4553-ad1c-25d9f2a3a630 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:03:16.318930 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-d9e1d733-cc38-4553-ad1c-25d9f2a3a630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:03:16.342381 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a38cb42-b0bb-4b50-a143-33c7ae13ddfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:16.343168 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a38cb42-b0bb-4b50-a143-33c7ae13ddfd None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:16.343828 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a38cb42-b0bb-4b50-a143-33c7ae13ddfd None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:16.344108 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a38cb42-b0bb-4b50-a143-33c7ae13ddfd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:16.344903 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a38cb42-b0bb-4b50-a143-33c7ae13ddfd None admin] RBAC: Adding query filter params (role.id=4eb30338d61e47808b51e050836643ea, scope.project.id=17dc3d39e2c048ca9070588f5717a578, user.id=58935367c0ad445f8477cdac0df04d0a) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:16.345216 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a38cb42-b0bb-4b50-a143-33c7ae13ddfd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:16.346780 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a38cb42-b0bb-4b50-a143-33c7ae13ddfd None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:16.352227 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 288/568] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Aug 4 21:03:16 2022] GET /identity/v3/role_assignments?user.id=58935367c0ad445f8477cdac0df04d0a&scope.project.id=17dc3d39e2c048ca9070588f5717a578&role.id=4eb30338d61e47808b51e050836643ea => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:16.352992 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a38cb42-b0bb-4b50-a143-33c7ae13ddfd None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:16.397502 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 281/569] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:03:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:03:16.398720 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e1d733-cc38-4553-ad1c-25d9f2a3a630 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:16.490111 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a38cb42-b0bb-4b50-a143-33c7ae13ddfd None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:16.519154 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e1d733-cc38-4553-ad1c-25d9f2a3a630 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:16.534298 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57623eaa-4f1e-4bf2-9ac0-8ccf7c5f266b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:16.534948 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57623eaa-4f1e-4bf2-9ac0-8ccf7c5f266b None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:16.535236 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57623eaa-4f1e-4bf2-9ac0-8ccf7c5f266b None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:16.535544 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57623eaa-4f1e-4bf2-9ac0-8ccf7c5f266b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:16.537419 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:16.547019 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57623eaa-4f1e-4bf2-9ac0-8ccf7c5f266b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:16.548403 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57623eaa-4f1e-4bf2-9ac0-8ccf7c5f266b None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:16.555915 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8a9f78-b4b6-44ca-8f8b-34c5ea8e0ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:16.556562 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8a9f78-b4b6-44ca-8f8b-34c5ea8e0ced None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:16.556852 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8a9f78-b4b6-44ca-8f8b-34c5ea8e0ced None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:16.557125 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8a9f78-b4b6-44ca-8f8b-34c5ea8e0ced None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:16.563704 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-57623eaa-4f1e-4bf2-9ac0-8ccf7c5f266b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:16.563704 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:16.563704 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:16.563704 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:16.563704 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:16.563704 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:16.563704 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:16.563704 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:16.563704 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:16.563704 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:16.563704 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:16.563704 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:16.563704 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:03:16.563704 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:03:16.563704 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:03:16.565340 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:03:16.565340 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:16.565340 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:16.565340 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:16.565340 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:16.565340 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:16.565340 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:16.565340 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:16.565340 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:16.565340 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:16.565340 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:16.565340 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:16.565340 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:16.565340 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:16.565340 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:16.565340 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:16.565340 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:16.566421 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:03:16.566421 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:03:16.566421 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:16.566421 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:16.567510 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 289/570] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:03:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:16.568734 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855c2d0d-fe0b-4e4a-8be8-a0aa3a805395 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:16.569091 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855c2d0d-fe0b-4e4a-8be8-a0aa3a805395 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:16.569365 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855c2d0d-fe0b-4e4a-8be8-a0aa3a805395 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:16.574542 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 290/571] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:03:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:03:16.574542 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855c2d0d-fe0b-4e4a-8be8-a0aa3a805395 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:16.576831 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8a9f78-b4b6-44ca-8f8b-34c5ea8e0ced None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:16.578323 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8a9f78-b4b6-44ca-8f8b-34c5ea8e0ced None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:16.581988 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:16.594285 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-3d8a9f78-b4b6-44ca-8f8b-34c5ea8e0ced None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:16.594285 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:16.594285 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:16.594285 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:16.594285 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:16.594285 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:16.594285 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:16.594285 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:16.594285 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:16.594285 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:16.594285 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:16.594285 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:16.594285 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:03:16.594285 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:03:16.594285 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:03:16.600745 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:03:16.600745 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:16.600745 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:16.600745 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:16.600745 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:16.600745 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:16.600745 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:16.600745 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:16.600745 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:16.600745 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:16.600745 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:16.600745 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:16.600745 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:16.600745 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:16.600745 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:16.600745 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:16.600745 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:16.601704 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:03:16.601704 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:03:16.601704 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:16.601704 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:16.601704 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 282/572] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:03:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:16.601704 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8a9f78-b4b6-44ca-8f8b-34c5ea8e0ced None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:16.692642 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855c2d0d-fe0b-4e4a-8be8-a0aa3a805395 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:16.702198 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8a9f78-b4b6-44ca-8f8b-34c5ea8e0ced None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:16.729961 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6dd358-eae8-4f58-af72-fef50593c573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:16.730697 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6dd358-eae8-4f58-af72-fef50593c573 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:16.731054 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6dd358-eae8-4f58-af72-fef50593c573 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:16.731432 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6dd358-eae8-4f58-af72-fef50593c573 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:16.737069 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792a9dd3-1969-4cb3-bb2d-4ab9ff9484ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:16.737683 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792a9dd3-1969-4cb3-bb2d-4ab9ff9484ef None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:16.737966 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792a9dd3-1969-4cb3-bb2d-4ab9ff9484ef None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:16.738234 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792a9dd3-1969-4cb3-bb2d-4ab9ff9484ef None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:16.738917 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792a9dd3-1969-4cb3-bb2d-4ab9ff9484ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:16.739220 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792a9dd3-1969-4cb3-bb2d-4ab9ff9484ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:16.741018 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792a9dd3-1969-4cb3-bb2d-4ab9ff9484ef None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:16.743599 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6dd358-eae8-4f58-af72-fef50593c573 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:03:16.748787 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-2d6dd358-eae8-4f58-af72-fef50593c573 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:03:16.748787 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:16.748787 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:16.748787 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:16.748787 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:16.748787 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:16.748787 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:16.748787 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:16.748787 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:16.748787 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:16.748787 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:16.748787 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:16.748787 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:03:16.748787 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:03:16.748787 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:03:16.748787 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:03:16.749917 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:03:16.749917 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:03:16.749917 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:03:16.749917 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:03:16.749917 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:16.749917 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:16.749917 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:16.749917 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:16.749917 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:16.749917 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:16.749917 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:16.749917 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:16.749917 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:16.749917 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:16.749917 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:16.749917 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:16.750878 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:16.750878 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:16.750878 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:16.750878 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:16.750878 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:03:16.750878 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:03:16.750878 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:03:16.750878 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:03:16.750878 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:03:16.750878 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:03:16.750878 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:03:16.750878 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:16.750878 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 291/573] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Aug 4 21:03:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:16.752579 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4074e677-ee2d-4132-81d0-0ae18f7eb602 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:16.752873 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4074e677-ee2d-4132-81d0-0ae18f7eb602 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:16.753149 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4074e677-ee2d-4132-81d0-0ae18f7eb602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:16.758846 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 283/574] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:03:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:16.760148 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792a9dd3-1969-4cb3-bb2d-4ab9ff9484ef None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:16.776270 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-4074e677-ee2d-4132-81d0-0ae18f7eb602 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:03:16.777555 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-4074e677-ee2d-4132-81d0-0ae18f7eb602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:03:16.794917 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:16.841776 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:16.850389 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 292/575] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:03:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:03:16.851207 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4074e677-ee2d-4132-81d0-0ae18f7eb602 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:16.882630 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792a9dd3-1969-4cb3-bb2d-4ab9ff9484ef None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:16.918837 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649b36e8-7276-486a-a69c-f56c810a181f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:16.919493 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649b36e8-7276-486a-a69c-f56c810a181f None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:16.919782 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649b36e8-7276-486a-a69c-f56c810a181f None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:16.920092 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649b36e8-7276-486a-a69c-f56c810a181f None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:16.920748 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649b36e8-7276-486a-a69c-f56c810a181f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:16.921060 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649b36e8-7276-486a-a69c-f56c810a181f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:16.922441 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649b36e8-7276-486a-a69c-f56c810a181f None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:16.934407 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 284/576] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:03:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:16.936622 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649b36e8-7276-486a-a69c-f56c810a181f None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:16.961940 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4074e677-ee2d-4132-81d0-0ae18f7eb602 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:17.001890 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3130c338-c9b6-46b9-abb2-f0008ca66be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:17.002533 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3130c338-c9b6-46b9-abb2-f0008ca66be6 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:17.002830 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3130c338-c9b6-46b9-abb2-f0008ca66be6 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:17.003086 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3130c338-c9b6-46b9-abb2-f0008ca66be6 None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:17.003895 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3130c338-c9b6-46b9-abb2-f0008ca66be6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:17.004205 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3130c338-c9b6-46b9-abb2-f0008ca66be6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:17.005773 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3130c338-c9b6-46b9-abb2-f0008ca66be6 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:17.019855 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 293/577] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Aug 4 21:03:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:17.020732 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3130c338-c9b6-46b9-abb2-f0008ca66be6 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:17.028910 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:17.041557 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649b36e8-7276-486a-a69c-f56c810a181f None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:17.085830 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33371395-140c-4315-aa74-6340e96b75cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:17.086479 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33371395-140c-4315-aa74-6340e96b75cc None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:17.086817 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33371395-140c-4315-aa74-6340e96b75cc None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:17.087454 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33371395-140c-4315-aa74-6340e96b75cc None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:17.089372 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:17.096896 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33371395-140c-4315-aa74-6340e96b75cc None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:17.098425 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33371395-140c-4315-aa74-6340e96b75cc None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:17.110007 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-33371395-140c-4315-aa74-6340e96b75cc None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 21:03:17.110007 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:17.110007 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:17.110007 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:17.110007 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:17.110007 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:17.110007 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:17.110007 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:17.110007 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:17.110007 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:17.110007 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:17.110007 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:17.110007 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 21:03:17.110007 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 21:03:17.110007 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 21:03:17.110007 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 21:03:17.116842 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:17.116842 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:17.116842 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 21:03:17.116842 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:03:17.116842 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 21:03:17.116842 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:03:17.116842 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:17.116842 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:17.116842 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:17.116842 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:17.116842 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:17.116842 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:17.116842 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:17.116842 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:17.116842 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:17.116842 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:17.116842 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:17.116842 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:17.117796 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:17.117796 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:17.117796 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 21:03:17.117796 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 21:03:17.117796 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 21:03:17.117796 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 21:03:17.117796 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 21:03:17.117796 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 21:03:17.117796 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 21:03:17.117796 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:17.117796 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 285/578] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Aug 4 21:03:16 2022] GET /identity/v3/users/cinder => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:17.117796 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33371395-140c-4315-aa74-6340e96b75cc None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:17.145200 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3130c338-c9b6-46b9-abb2-f0008ca66be6 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:17.181901 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43656b8f-fe7b-4c2a-b23e-7cb612e97461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:17.182545 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43656b8f-fe7b-4c2a-b23e-7cb612e97461 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:17.182834 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43656b8f-fe7b-4c2a-b23e-7cb612e97461 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:17.183110 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43656b8f-fe7b-4c2a-b23e-7cb612e97461 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:17.193583 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43656b8f-fe7b-4c2a-b23e-7cb612e97461 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:03:17.198969 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-43656b8f-fe7b-4c2a-b23e-7cb612e97461 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:03:17.198969 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:17.198969 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:17.198969 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:17.198969 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:17.198969 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:17.198969 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:17.198969 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:17.198969 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:17.198969 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:17.198969 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:17.198969 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:17.198969 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:03:17.198969 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:03:17.198969 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:03:17.198969 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:03:17.199975 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:03:17.199975 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:03:17.199975 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:03:17.199975 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:03:17.199975 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:17.199975 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:17.199975 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:17.199975 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:17.199975 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:17.199975 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:17.199975 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:17.199975 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:17.199975 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:17.199975 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:17.199975 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:17.199975 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:17.200788 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:17.200788 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:17.200788 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:17.200788 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:17.200788 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:03:17.200788 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:03:17.200788 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:03:17.200788 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:03:17.200788 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:03:17.200788 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:03:17.200788 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:03:17.200788 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:17.200788 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 294/579] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Aug 4 21:03:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:17.201491 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43656b8f-fe7b-4c2a-b23e-7cb612e97461 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:17.224461 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33371395-140c-4315-aa74-6340e96b75cc None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:17.259857 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ce8365-e6df-4526-b07e-08452c9eecf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:17.260496 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ce8365-e6df-4526-b07e-08452c9eecf3 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:17.260781 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ce8365-e6df-4526-b07e-08452c9eecf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:17.261056 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ce8365-e6df-4526-b07e-08452c9eecf3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:17.268938 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ce8365-e6df-4526-b07e-08452c9eecf3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:17.270393 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ce8365-e6df-4526-b07e-08452c9eecf3 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:17.283038 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-88ce8365-e6df-4526-b07e-08452c9eecf3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:03:17.283038 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:17.283038 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:17.283038 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:17.283038 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:17.283038 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:17.283038 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:17.283038 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:17.283038 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:17.283038 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:17.283038 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:17.283038 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:17.283038 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 21:03:17.283038 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 21:03:17.283038 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 21:03:17.284312 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 21:03:17.284312 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:17.284312 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:17.284312 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:17.284312 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:17.284312 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:17.284312 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:17.284312 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:17.284312 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:17.284312 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:17.284312 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:17.284312 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:17.284312 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:17.284312 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:17.284312 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:17.284312 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:17.284312 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:17.285190 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 21:03:17.285190 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 21:03:17.285190 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 21:03:17.285190 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 21:03:17.285190 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 21:03:17.285190 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 21:03:17.285190 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:03:17.285190 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:17.285190 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 286/580] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Aug 4 21:03:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:17.285766 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ce8365-e6df-4526-b07e-08452c9eecf3 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:17.296510 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:17.316523 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43656b8f-fe7b-4c2a-b23e-7cb612e97461 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:17.335048 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:17.352427 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8117c8-aa67-4c75-8974-5b4ee8c2ab36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:17.353077 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8117c8-aa67-4c75-8974-5b4ee8c2ab36 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:17.353377 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8117c8-aa67-4c75-8974-5b4ee8c2ab36 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:17.353652 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8117c8-aa67-4c75-8974-5b4ee8c2ab36 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:17.364742 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8117c8-aa67-4c75-8974-5b4ee8c2ab36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:17.366775 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8117c8-aa67-4c75-8974-5b4ee8c2ab36 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:17.382933 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-4d8117c8-aa67-4c75-8974-5b4ee8c2ab36 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:17.382933 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:17.382933 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:17.382933 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:17.382933 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:17.382933 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:17.382933 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:17.382933 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:17.382933 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:17.382933 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:17.382933 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:17.382933 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:17.382933 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:03:17.382933 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:03:17.382933 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:03:17.384011 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:03:17.384011 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:17.384011 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:17.384011 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:17.384011 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:17.384011 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:17.384011 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:17.384011 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:17.384011 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:17.384011 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:17.384011 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:17.384011 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:17.384011 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:17.384011 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:17.384011 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:17.384011 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:17.384011 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:17.384902 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:03:17.384902 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:03:17.384902 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:17.384902 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:17.384902 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 295/581] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:03:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:17.385870 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8117c8-aa67-4c75-8974-5b4ee8c2ab36 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:17.391816 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ce8365-e6df-4526-b07e-08452c9eecf3 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:17.427307 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e81c32-158d-4d0c-b563-3ac122293b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:17.427954 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e81c32-158d-4d0c-b563-3ac122293b3f None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:17.428245 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e81c32-158d-4d0c-b563-3ac122293b3f None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:17.428572 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e81c32-158d-4d0c-b563-3ac122293b3f None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:17.429369 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e81c32-158d-4d0c-b563-3ac122293b3f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:17.429682 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e81c32-158d-4d0c-b563-3ac122293b3f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:17.431143 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e81c32-158d-4d0c-b563-3ac122293b3f None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:17.450870 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 287/582] 10.222.0.25 () {56 vars in 1137 bytes} [Thu Aug 4 21:03:17 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:17.456646 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e81c32-158d-4d0c-b563-3ac122293b3f None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:17.504754 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8117c8-aa67-4c75-8974-5b4ee8c2ab36 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:17.530001 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:17.542912 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1924b9dd-4898-48c1-9167-4636fd3b02d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:17.542912 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1924b9dd-4898-48c1-9167-4636fd3b02d2 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:17.542912 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1924b9dd-4898-48c1-9167-4636fd3b02d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:17.542912 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1924b9dd-4898-48c1-9167-4636fd3b02d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:17.543266 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1924b9dd-4898-48c1-9167-4636fd3b02d2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:17.543266 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1924b9dd-4898-48c1-9167-4636fd3b02d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:17.543266 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1924b9dd-4898-48c1-9167-4636fd3b02d2 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:17.555720 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 296/583] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Aug 4 21:03:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:17.555720 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1924b9dd-4898-48c1-9167-4636fd3b02d2 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:17.566271 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e81c32-158d-4d0c-b563-3ac122293b3f None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:17.586531 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:17.603094 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01776799-93a6-4077-912d-e97c19ee624a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:17.603747 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01776799-93a6-4077-912d-e97c19ee624a None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:17.604031 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01776799-93a6-4077-912d-e97c19ee624a None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:17.604303 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01776799-93a6-4077-912d-e97c19ee624a None admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:17.605050 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01776799-93a6-4077-912d-e97c19ee624a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:17.605360 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01776799-93a6-4077-912d-e97c19ee624a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:17.606814 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01776799-93a6-4077-912d-e97c19ee624a None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:17.620885 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 288/584] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Aug 4 21:03:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:17.622487 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01776799-93a6-4077-912d-e97c19ee624a None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:17.663668 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1924b9dd-4898-48c1-9167-4636fd3b02d2 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:17.699197 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae57517-41dd-46fd-9d3d-6b395178d5c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:17.699965 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae57517-41dd-46fd-9d3d-6b395178d5c9 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:17.700380 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae57517-41dd-46fd-9d3d-6b395178d5c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:17.700737 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae57517-41dd-46fd-9d3d-6b395178d5c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:17.701879 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae57517-41dd-46fd-9d3d-6b395178d5c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:17.702251 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae57517-41dd-46fd-9d3d-6b395178d5c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:17.703774 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae57517-41dd-46fd-9d3d-6b395178d5c9 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:17.715903 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 297/585] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:03:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:17.720687 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae57517-41dd-46fd-9d3d-6b395178d5c9 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:17.729122 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01776799-93a6-4077-912d-e97c19ee624a None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:17.762103 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:17.773003 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa850c7-735c-4d27-a4c5-e7bfc03593a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:17.777498 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa850c7-735c-4d27-a4c5-e7bfc03593a3 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:17.777498 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa850c7-735c-4d27-a4c5-e7bfc03593a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:17.777498 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa850c7-735c-4d27-a4c5-e7bfc03593a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:17.784505 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa850c7-735c-4d27-a4c5-e7bfc03593a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:17.785972 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa850c7-735c-4d27-a4c5-e7bfc03593a3 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:17.810398 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-3aa850c7-735c-4d27-a4c5-e7bfc03593a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:17.810398 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:17.810398 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:17.810398 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:17.810398 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:17.810398 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:17.810398 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:17.810398 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:17.810398 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:17.810398 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:17.810398 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:17.810398 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:17.810398 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:03:17.810398 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:03:17.810398 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:03:17.813166 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:03:17.813166 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:17.813166 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:17.813166 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:17.813166 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:17.813166 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:17.813166 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:17.813166 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:17.813166 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:17.813166 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:17.813166 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:17.813166 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:17.813166 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:17.813166 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:17.813166 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:17.813166 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:17.813166 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:17.814048 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:03:17.814048 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:03:17.814048 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:17.814048 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:17.814048 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 289/586] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:03:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:17.814048 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa850c7-735c-4d27-a4c5-e7bfc03593a3 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:17.824541 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:17.835435 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae57517-41dd-46fd-9d3d-6b395178d5c9 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:17.870617 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f77d0ce-2913-496b-872e-04e76c88c18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:17.871265 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f77d0ce-2913-496b-872e-04e76c88c18b None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:17.871605 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f77d0ce-2913-496b-872e-04e76c88c18b None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:17.871884 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f77d0ce-2913-496b-872e-04e76c88c18b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:17.883044 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f77d0ce-2913-496b-872e-04e76c88c18b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:17.884534 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f77d0ce-2913-496b-872e-04e76c88c18b None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:17.899669 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-2f77d0ce-2913-496b-872e-04e76c88c18b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:17.899669 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:17.899669 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:17.899669 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:17.899669 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:17.899669 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:17.899669 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:17.899669 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:17.899669 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:17.899669 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:17.899669 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:17.899669 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:17.899669 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:03:17.899669 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:03:17.899669 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:03:17.900622 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:03:17.900622 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:17.900622 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:17.900622 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:17.900622 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:17.900622 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:17.900622 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:17.900622 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:17.900622 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:17.900622 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:17.900622 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:17.900622 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:17.900622 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:17.900622 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:17.900622 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:17.900622 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:17.900622 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:17.901508 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:03:17.901508 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:03:17.901508 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:17.901508 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:17.901508 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 298/587] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:03:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:17.901791 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f77d0ce-2913-496b-872e-04e76c88c18b None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:17.929566 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa850c7-735c-4d27-a4c5-e7bfc03593a3 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:17.967120 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152bbbbb-033f-4513-b60b-8c3b88d0d150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:17.967821 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152bbbbb-033f-4513-b60b-8c3b88d0d150 None admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:17.968105 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152bbbbb-033f-4513-b60b-8c3b88d0d150 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:17.968376 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152bbbbb-033f-4513-b60b-8c3b88d0d150 None admin] PATH_INFO: `/v3/projects/17dc3d39e2c048ca9070588f5717a578/users/6a2ce1df87fa47e5be301916bf3245fb/roles/4eb30338d61e47808b51e050836643ea` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:17.984564 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152bbbbb-033f-4513-b60b-8c3b88d0d150 None admin] RBAC: Authorizing `identity:create_grant(project_id=17dc3d39e2c048ca9070588f5717a578, user_id=6a2ce1df87fa47e5be301916bf3245fb, role_id=4eb30338d61e47808b51e050836643ea)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:17.987457 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152bbbbb-033f-4513-b60b-8c3b88d0d150 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:18.010441 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f77d0ce-2913-496b-872e-04e76c88c18b None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:18.043494 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d44385-0dd5-494a-ace9-43d46630dbba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:18.044035 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d44385-0dd5-494a-ace9-43d46630dbba None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:18.044326 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d44385-0dd5-494a-ace9-43d46630dbba None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:18.044600 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d44385-0dd5-494a-ace9-43d46630dbba None admin] PATH_INFO: `/v3/users/glance` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:18.052624 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d44385-0dd5-494a-ace9-43d46630dbba None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:18.054064 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d44385-0dd5-494a-ace9-43d46630dbba None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:18.065464 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-52d44385-0dd5-494a-ace9-43d46630dbba None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 04 21:03:18.065464 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:18.065464 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:18.065464 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:18.065464 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:18.065464 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:18.065464 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:18.065464 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:18.065464 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:18.065464 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:18.065464 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:18.065464 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:18.065464 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 21:03:18.065464 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 21:03:18.065464 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 21:03:18.065464 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 21:03:18.066566 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:18.066566 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:18.066566 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 21:03:18.066566 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:03:18.066566 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 21:03:18.066566 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:03:18.066566 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:18.066566 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:18.066566 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:18.066566 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:18.066566 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:18.066566 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:18.066566 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:18.066566 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:18.066566 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:18.066566 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:18.066566 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:18.066566 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:18.067628 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:18.067628 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:18.067628 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 21:03:18.067628 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 21:03:18.067628 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 21:03:18.067628 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 21:03:18.067628 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 21:03:18.067628 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 21:03:18.067628 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 04 21:03:18.067628 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:18.067628 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 299/588] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Aug 4 21:03:17 2022] GET /identity/v3/users/glance => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:18.067628 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d44385-0dd5-494a-ace9-43d46630dbba None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:18.074161 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:18.111352 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 290/589] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Aug 4 21:03:17 2022] PUT /identity/v3/projects/17dc3d39e2c048ca9070588f5717a578/users/6a2ce1df87fa47e5be301916bf3245fb/roles/4eb30338d61e47808b51e050836643ea => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:03:18.111352 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152bbbbb-033f-4513-b60b-8c3b88d0d150 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:18.145953 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:18.209188 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d44385-0dd5-494a-ace9-43d46630dbba None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:18.249535 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca6afc8-0d0b-4711-9fba-d59195250df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:18.250282 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca6afc8-0d0b-4711-9fba-d59195250df0 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:18.250666 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca6afc8-0d0b-4711-9fba-d59195250df0 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:18.251044 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca6afc8-0d0b-4711-9fba-d59195250df0 None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:18.251955 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca6afc8-0d0b-4711-9fba-d59195250df0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:18.252387 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca6afc8-0d0b-4711-9fba-d59195250df0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:18.253936 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca6afc8-0d0b-4711-9fba-d59195250df0 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:18.259074 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152bbbbb-033f-4513-b60b-8c3b88d0d150 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:18.269769 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 300/590] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:03:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:18.270766 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca6afc8-0d0b-4711-9fba-d59195250df0 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:18.300592 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7f3c21-0757-4a9d-99b2-2da9d7e28c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:18.301536 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7f3c21-0757-4a9d-99b2-2da9d7e28c3b None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:18.301967 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7f3c21-0757-4a9d-99b2-2da9d7e28c3b None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:18.302237 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7f3c21-0757-4a9d-99b2-2da9d7e28c3b None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:18.302941 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7f3c21-0757-4a9d-99b2-2da9d7e28c3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:18.303244 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7f3c21-0757-4a9d-99b2-2da9d7e28c3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:18.304705 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7f3c21-0757-4a9d-99b2-2da9d7e28c3b None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:18.318220 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 291/591] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:03:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:18.319033 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7f3c21-0757-4a9d-99b2-2da9d7e28c3b None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:18.354076 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:18.396272 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca6afc8-0d0b-4711-9fba-d59195250df0 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:18.418314 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:18.430253 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7f3c21-0757-4a9d-99b2-2da9d7e28c3b None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:18.433301 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df43489c-e7bc-4a55-9a15-cdcbe371f16f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:18.433923 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df43489c-e7bc-4a55-9a15-cdcbe371f16f None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:18.434262 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df43489c-e7bc-4a55-9a15-cdcbe371f16f None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:18.434535 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df43489c-e7bc-4a55-9a15-cdcbe371f16f None admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:18.435336 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df43489c-e7bc-4a55-9a15-cdcbe371f16f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:18.435653 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df43489c-e7bc-4a55-9a15-cdcbe371f16f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:18.437070 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df43489c-e7bc-4a55-9a15-cdcbe371f16f None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:18.459557 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 301/592] 10.222.0.25 () {56 vars in 1137 bytes} [Thu Aug 4 21:03:18 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:18.459557 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df43489c-e7bc-4a55-9a15-cdcbe371f16f None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:18.473116 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34727ad7-41e1-455b-bda1-3262e683da22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:18.473768 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34727ad7-41e1-455b-bda1-3262e683da22 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:18.474057 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34727ad7-41e1-455b-bda1-3262e683da22 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:18.474332 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34727ad7-41e1-455b-bda1-3262e683da22 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:18.481615 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34727ad7-41e1-455b-bda1-3262e683da22 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:18.483073 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34727ad7-41e1-455b-bda1-3262e683da22 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:18.495705 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-34727ad7-41e1-455b-bda1-3262e683da22 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:03:18.495705 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:18.495705 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:18.495705 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:18.495705 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:18.495705 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:18.495705 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:18.495705 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:18.495705 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:18.495705 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:18.495705 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:18.495705 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:18.495705 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 21:03:18.495705 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 21:03:18.495705 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 21:03:18.496627 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 21:03:18.496627 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:18.496627 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:18.496627 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:18.496627 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:18.496627 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:18.496627 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:18.496627 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:18.496627 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:18.496627 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:18.496627 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:18.496627 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:18.496627 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:18.496627 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:18.496627 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:18.496627 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:18.496627 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:18.497574 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 21:03:18.497574 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 21:03:18.497574 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 21:03:18.497574 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 21:03:18.497574 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 21:03:18.497574 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 21:03:18.497574 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:03:18.497574 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:18.499454 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 292/593] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Aug 4 21:03:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:18.500676 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635ee8e1-4e0a-4c11-a39a-879e6d534617 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:18.500963 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635ee8e1-4e0a-4c11-a39a-879e6d534617 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:18.501347 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635ee8e1-4e0a-4c11-a39a-879e6d534617 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:18.502682 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 293/594] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:03:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:03:18.503411 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635ee8e1-4e0a-4c11-a39a-879e6d534617 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:18.573577 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df43489c-e7bc-4a55-9a15-cdcbe371f16f None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:18.598549 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:18.610078 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47fb4c0-5d01-47d3-956e-bd60e87ad46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:18.610918 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47fb4c0-5d01-47d3-956e-bd60e87ad46d None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:18.611276 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47fb4c0-5d01-47d3-956e-bd60e87ad46d None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:18.611832 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47fb4c0-5d01-47d3-956e-bd60e87ad46d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:18.618001 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635ee8e1-4e0a-4c11-a39a-879e6d534617 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:18.619530 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47fb4c0-5d01-47d3-956e-bd60e87ad46d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:18.620986 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47fb4c0-5d01-47d3-956e-bd60e87ad46d None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:18.632696 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-b47fb4c0-5d01-47d3-956e-bd60e87ad46d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 04 21:03:18.632696 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:18.632696 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:18.632696 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:18.632696 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:18.632696 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:18.632696 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:18.632696 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:18.632696 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:18.632696 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:18.632696 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:18.632696 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:18.632696 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 21:03:18.632696 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 21:03:18.632696 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 21:03:18.632696 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 21:03:18.633693 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:18.633693 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:18.633693 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 21:03:18.633693 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:03:18.633693 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 21:03:18.633693 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:03:18.633693 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:18.633693 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:18.633693 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:18.633693 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:18.633693 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:18.633693 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:18.633693 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:18.633693 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:18.633693 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:18.633693 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:18.633693 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:18.633693 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:18.634670 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:18.634670 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:18.634670 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 21:03:18.634670 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 21:03:18.634670 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 21:03:18.634670 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 21:03:18.634670 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 21:03:18.634670 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 21:03:18.634670 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 04 21:03:18.634670 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:18.634670 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 302/595] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Aug 4 21:03:18 2022] GET /identity/v3/users/neutron => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:18.634670 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47fb4c0-5d01-47d3-956e-bd60e87ad46d None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:18.663093 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d11dd2-22d1-40e6-98d0-9249a0c6de16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:18.663759 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d11dd2-22d1-40e6-98d0-9249a0c6de16 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:18.664044 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d11dd2-22d1-40e6-98d0-9249a0c6de16 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:18.664307 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d11dd2-22d1-40e6-98d0-9249a0c6de16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:18.675482 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d11dd2-22d1-40e6-98d0-9249a0c6de16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:18.676985 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d11dd2-22d1-40e6-98d0-9249a0c6de16 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:18.687147 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:18.694966 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-80d11dd2-22d1-40e6-98d0-9249a0c6de16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:18.694966 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:18.694966 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:18.694966 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:18.694966 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:18.694966 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:18.694966 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:18.694966 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:18.694966 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:18.694966 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:18.694966 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:18.694966 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:18.694966 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:03:18.694966 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:03:18.694966 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:03:18.705861 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:03:18.705861 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:18.705861 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:18.705861 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:18.705861 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:18.705861 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:18.705861 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:18.705861 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:18.705861 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:18.705861 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:18.705861 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:18.705861 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:18.705861 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:18.705861 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:18.705861 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:18.705861 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:18.705861 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:18.706950 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:03:18.706950 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:03:18.706950 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:18.706950 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:18.706950 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 294/596] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:03:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:18.706950 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fd1c91-c69d-49ca-8611-824ef50de358 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:18.706950 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fd1c91-c69d-49ca-8611-824ef50de358 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:18.706950 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fd1c91-c69d-49ca-8611-824ef50de358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:18.718642 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-a2fd1c91-c69d-49ca-8611-824ef50de358 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:03:18.719845 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-a2fd1c91-c69d-49ca-8611-824ef50de358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:03:18.756177 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47fb4c0-5d01-47d3-956e-bd60e87ad46d None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:18.795383 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 295/597] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:03:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:03:18.795383 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fd1c91-c69d-49ca-8611-824ef50de358 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:18.805605 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81109202-9627-42ac-88f1-a2d325d0d353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:18.806755 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81109202-9627-42ac-88f1-a2d325d0d353 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:18.807214 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81109202-9627-42ac-88f1-a2d325d0d353 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:18.807940 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81109202-9627-42ac-88f1-a2d325d0d353 None admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:18.809399 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81109202-9627-42ac-88f1-a2d325d0d353 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:18.809907 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81109202-9627-42ac-88f1-a2d325d0d353 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:18.812027 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81109202-9627-42ac-88f1-a2d325d0d353 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:18.828603 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 303/598] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Aug 4 21:03:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:18.829640 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81109202-9627-42ac-88f1-a2d325d0d353 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:18.878092 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:18.898880 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fd1c91-c69d-49ca-8611-824ef50de358 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:18.928807 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:18.935406 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad584ee3-20cd-4992-9359-27e144974f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:18.936034 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad584ee3-20cd-4992-9359-27e144974f94 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:18.936314 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad584ee3-20cd-4992-9359-27e144974f94 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:18.936703 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad584ee3-20cd-4992-9359-27e144974f94 None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:18.937389 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81109202-9627-42ac-88f1-a2d325d0d353 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:18.937578 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad584ee3-20cd-4992-9359-27e144974f94 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:18.937881 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad584ee3-20cd-4992-9359-27e144974f94 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:18.939331 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad584ee3-20cd-4992-9359-27e144974f94 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:18.960600 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 296/599] 10.222.0.25 () {56 vars in 1139 bytes} [Thu Aug 4 21:03:18 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:18.960600 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad584ee3-20cd-4992-9359-27e144974f94 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:18.979627 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eddcf5-c082-4c28-96e8-dab0a2991a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:18.980275 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eddcf5-c082-4c28-96e8-dab0a2991a46 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:18.980537 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eddcf5-c082-4c28-96e8-dab0a2991a46 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:18.980807 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eddcf5-c082-4c28-96e8-dab0a2991a46 None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:18.981498 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29eddcf5-c082-4c28-96e8-dab0a2991a46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:18.981801 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29eddcf5-c082-4c28-96e8-dab0a2991a46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:18.983170 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29eddcf5-c082-4c28-96e8-dab0a2991a46 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:18.995338 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 304/600] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:03:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:19.000031 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eddcf5-c082-4c28-96e8-dab0a2991a46 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:19.069350 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad584ee3-20cd-4992-9359-27e144974f94 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:19.109770 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eddcf5-c082-4c28-96e8-dab0a2991a46 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:19.110447 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2f2cb7-8b2b-4964-a7ef-bcb92cd5106c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:19.111094 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2f2cb7-8b2b-4964-a7ef-bcb92cd5106c None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:19.111974 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2f2cb7-8b2b-4964-a7ef-bcb92cd5106c None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:19.112247 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2f2cb7-8b2b-4964-a7ef-bcb92cd5106c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:19.113167 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:19.122422 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2f2cb7-8b2b-4964-a7ef-bcb92cd5106c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:03:19.127370 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-ec2f2cb7-8b2b-4964-a7ef-bcb92cd5106c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:03:19.127370 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:19.127370 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:19.127370 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:19.127370 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:19.127370 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:19.127370 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:19.127370 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:19.127370 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:19.127370 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:19.127370 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:19.127370 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:19.127370 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:03:19.127370 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:03:19.127370 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:03:19.127370 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:03:19.131699 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:03:19.131699 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:03:19.131699 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:03:19.131699 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:03:19.131699 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:19.131699 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:19.131699 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:19.131699 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:19.131699 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:19.131699 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:19.131699 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:19.131699 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:19.131699 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:19.131699 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:19.131699 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:19.131699 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:19.132521 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:19.132521 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:19.132521 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:19.132521 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:19.132521 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:03:19.132521 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:03:19.132521 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:03:19.132521 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:03:19.132521 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:03:19.132521 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:03:19.132521 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:03:19.132521 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:19.133362 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 297/601] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Aug 4 21:03:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:19.134201 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2f2cb7-8b2b-4964-a7ef-bcb92cd5106c None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:19.153267 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23244a47-d25a-4189-8f80-1d192c7bd7f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:19.153925 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23244a47-d25a-4189-8f80-1d192c7bd7f7 None admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:19.154204 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23244a47-d25a-4189-8f80-1d192c7bd7f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:19.154524 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23244a47-d25a-4189-8f80-1d192c7bd7f7 None admin] PATH_INFO: `/v3/projects/17dc3d39e2c048ca9070588f5717a578/users/e31f2728056b460fa474478eb85ca1d8/roles/4eb30338d61e47808b51e050836643ea` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:19.170835 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23244a47-d25a-4189-8f80-1d192c7bd7f7 None admin] RBAC: Authorizing `identity:create_grant(project_id=17dc3d39e2c048ca9070588f5717a578, user_id=e31f2728056b460fa474478eb85ca1d8, role_id=4eb30338d61e47808b51e050836643ea)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:19.172288 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23244a47-d25a-4189-8f80-1d192c7bd7f7 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:19.183544 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:19.238622 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2f2cb7-8b2b-4964-a7ef-bcb92cd5106c None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:19.239840 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 305/602] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Aug 4 21:03:18 2022] PUT /identity/v3/projects/17dc3d39e2c048ca9070588f5717a578/users/e31f2728056b460fa474478eb85ca1d8/roles/4eb30338d61e47808b51e050836643ea => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:03:19.252465 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23244a47-d25a-4189-8f80-1d192c7bd7f7 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:19.280608 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa22bf30-fa2a-42a6-9a2c-373d88c1d77b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:19.281270 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa22bf30-fa2a-42a6-9a2c-373d88c1d77b None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:19.281568 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa22bf30-fa2a-42a6-9a2c-373d88c1d77b None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:19.281903 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa22bf30-fa2a-42a6-9a2c-373d88c1d77b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:19.299241 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa22bf30-fa2a-42a6-9a2c-373d88c1d77b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:19.300925 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa22bf30-fa2a-42a6-9a2c-373d88c1d77b None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:19.318330 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-aa22bf30-fa2a-42a6-9a2c-373d88c1d77b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:19.318330 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:19.318330 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:19.318330 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:19.318330 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:19.318330 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:19.318330 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:19.318330 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:19.318330 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:19.318330 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:19.318330 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:19.318330 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:19.318330 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:03:19.318330 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:03:19.318330 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:03:19.319341 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:03:19.319341 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:19.319341 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:19.319341 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:19.319341 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:19.319341 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:19.319341 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:19.319341 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:19.319341 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:19.319341 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:19.319341 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:19.319341 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:19.319341 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:19.319341 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:19.319341 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:19.319341 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:19.319341 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:19.320280 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:03:19.320280 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:03:19.320280 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:19.320280 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:19.320280 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 298/603] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:03:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:19.321011 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa22bf30-fa2a-42a6-9a2c-373d88c1d77b None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:19.403262 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23244a47-d25a-4189-8f80-1d192c7bd7f7 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:19.448375 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef5f197-fd25-4dd3-a379-c353eb5ea08c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:19.448902 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:19.449427 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef5f197-fd25-4dd3-a379-c353eb5ea08c None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:19.449688 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef5f197-fd25-4dd3-a379-c353eb5ea08c None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:19.449957 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef5f197-fd25-4dd3-a379-c353eb5ea08c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:19.453869 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa22bf30-fa2a-42a6-9a2c-373d88c1d77b None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:19.464675 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef5f197-fd25-4dd3-a379-c353eb5ea08c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:19.465993 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef5f197-fd25-4dd3-a379-c353eb5ea08c None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:19.484247 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-bef5f197-fd25-4dd3-a379-c353eb5ea08c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:03:19.484247 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:19.484247 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:19.484247 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:19.484247 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:19.484247 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:19.484247 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:19.484247 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:19.484247 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:19.484247 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:19.484247 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:19.484247 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:19.484247 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 21:03:19.484247 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 21:03:19.484247 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 21:03:19.485217 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 21:03:19.485217 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:19.485217 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:19.485217 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:19.485217 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:19.485217 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:19.485217 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:19.485217 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:19.485217 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:19.485217 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:19.485217 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:19.485217 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:19.485217 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:19.485217 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:19.485217 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:19.485217 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:19.485217 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:19.487066 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 21:03:19.487066 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 21:03:19.487066 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 21:03:19.487066 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 21:03:19.487066 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 21:03:19.487066 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 21:03:19.487066 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:03:19.487066 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:19.487066 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 306/604] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Aug 4 21:03:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:19.487589 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef5f197-fd25-4dd3-a379-c353eb5ea08c None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:19.496524 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf66020-dd8a-48f4-89ad-1689bbf71bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:19.497543 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf66020-dd8a-48f4-89ad-1689bbf71bf6 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:19.498018 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf66020-dd8a-48f4-89ad-1689bbf71bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:19.498482 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf66020-dd8a-48f4-89ad-1689bbf71bf6 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:19.499575 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf66020-dd8a-48f4-89ad-1689bbf71bf6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:19.500183 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf66020-dd8a-48f4-89ad-1689bbf71bf6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:19.501781 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf66020-dd8a-48f4-89ad-1689bbf71bf6 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:19.512163 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 299/605] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Aug 4 21:03:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:19.512940 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:19.514201 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf66020-dd8a-48f4-89ad-1689bbf71bf6 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:19.604959 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef5f197-fd25-4dd3-a379-c353eb5ea08c None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:19.620785 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf66020-dd8a-48f4-89ad-1689bbf71bf6 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:19.641854 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1e2b8a-6a6e-4c54-b1d7-92971ed3c8fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:19.642777 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1e2b8a-6a6e-4c54-b1d7-92971ed3c8fe None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:19.642777 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1e2b8a-6a6e-4c54-b1d7-92971ed3c8fe None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:19.643070 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1e2b8a-6a6e-4c54-b1d7-92971ed3c8fe None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:19.643807 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1e2b8a-6a6e-4c54-b1d7-92971ed3c8fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:19.644116 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1e2b8a-6a6e-4c54-b1d7-92971ed3c8fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:19.645501 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1e2b8a-6a6e-4c54-b1d7-92971ed3c8fe None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:19.663021 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 307/606] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:03:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:19.663021 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1e2b8a-6a6e-4c54-b1d7-92971ed3c8fe None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:19.663793 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc9c72d-f271-4497-9de8-fedd3cd5773e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:19.664187 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc9c72d-f271-4497-9de8-fedd3cd5773e None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:19.664467 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc9c72d-f271-4497-9de8-fedd3cd5773e None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:19.664737 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc9c72d-f271-4497-9de8-fedd3cd5773e None admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:19.665488 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc9c72d-f271-4497-9de8-fedd3cd5773e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:19.665795 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc9c72d-f271-4497-9de8-fedd3cd5773e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:19.667695 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc9c72d-f271-4497-9de8-fedd3cd5773e None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:19.687400 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 300/607] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Aug 4 21:03:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:19.688659 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc9c72d-f271-4497-9de8-fedd3cd5773e None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:19.722468 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:19.770549 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:19.811156 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1e2b8a-6a6e-4c54-b1d7-92971ed3c8fe None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:19.824891 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc9c72d-f271-4497-9de8-fedd3cd5773e None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:19.858164 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb3fc72-fe3e-44db-9580-646c89c53200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:19.859847 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb3fc72-fe3e-44db-9580-646c89c53200 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:19.860519 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb3fc72-fe3e-44db-9580-646c89c53200 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:19.861122 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb3fc72-fe3e-44db-9580-646c89c53200 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:19.868669 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f15e4ab-2b9e-47c4-a0e7-3101d9d0ed2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:19.869322 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f15e4ab-2b9e-47c4-a0e7-3101d9d0ed2d None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:19.869606 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f15e4ab-2b9e-47c4-a0e7-3101d9d0ed2d None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:19.869881 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f15e4ab-2b9e-47c4-a0e7-3101d9d0ed2d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:19.876280 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb3fc72-fe3e-44db-9580-646c89c53200 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:19.878795 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f15e4ab-2b9e-47c4-a0e7-3101d9d0ed2d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:19.879212 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb3fc72-fe3e-44db-9580-646c89c53200 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:19.880514 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f15e4ab-2b9e-47c4-a0e7-3101d9d0ed2d None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:19.894005 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-3f15e4ab-2b9e-47c4-a0e7-3101d9d0ed2d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:03:19.894005 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:19.894005 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:19.894005 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:19.894005 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:19.894005 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:19.894005 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:19.894005 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:19.894005 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:19.894005 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:19.894005 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:19.894005 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:19.894005 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 21:03:19.894005 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 21:03:19.894005 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 21:03:19.894954 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 21:03:19.894954 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:19.894954 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:19.894954 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:19.894954 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:19.894954 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:19.894954 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:19.894954 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:19.894954 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:19.894954 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:19.894954 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:19.894954 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:19.894954 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:19.894954 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:19.894954 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:19.894954 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:19.894954 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:19.896415 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 21:03:19.896415 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 21:03:19.896415 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 21:03:19.896415 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 21:03:19.896415 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 21:03:19.896415 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 21:03:19.896415 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:03:19.896415 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:19.899144 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-2bb3fc72-fe3e-44db-9580-646c89c53200 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:19.899144 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:19.899144 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:19.899144 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:19.899144 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:19.899144 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:19.899144 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:19.899144 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:19.899144 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:19.899144 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:19.899144 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:19.899144 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:19.899144 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:03:19.899144 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:03:19.899144 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:03:19.900400 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:03:19.900400 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:19.900400 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:19.900400 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:19.900400 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:19.900400 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:19.900400 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:19.900400 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:19.900400 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:19.900400 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:19.900400 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:19.900400 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:19.900400 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:19.900400 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:19.900400 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:19.900400 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:19.900400 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:19.901272 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:03:19.901272 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:03:19.901272 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:19.901272 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:19.901487 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 301/608] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Aug 4 21:03:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:19.901487 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f15e4ab-2b9e-47c4-a0e7-3101d9d0ed2d None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:19.903936 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 308/609] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:03:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:19.906055 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb3fc72-fe3e-44db-9580-646c89c53200 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:19.994589 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:20.014494 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb3fc72-fe3e-44db-9580-646c89c53200 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:20.017428 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f15e4ab-2b9e-47c4-a0e7-3101d9d0ed2d None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:20.039976 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:20.052137 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb7da5a-66c2-4fa3-857a-e9a160850d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:20.052713 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb7da5a-66c2-4fa3-857a-e9a160850d05 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:20.053002 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb7da5a-66c2-4fa3-857a-e9a160850d05 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:20.053279 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb7da5a-66c2-4fa3-857a-e9a160850d05 None admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:20.054040 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb7da5a-66c2-4fa3-857a-e9a160850d05 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:20.054411 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb7da5a-66c2-4fa3-857a-e9a160850d05 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:20.055922 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb7da5a-66c2-4fa3-857a-e9a160850d05 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:20.060865 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7894835-46fe-49df-bce8-06b37990e224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:20.061828 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7894835-46fe-49df-bce8-06b37990e224 None admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:20.062357 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7894835-46fe-49df-bce8-06b37990e224 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:20.062838 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7894835-46fe-49df-bce8-06b37990e224 None admin] PATH_INFO: `/v3/projects/17dc3d39e2c048ca9070588f5717a578/users/f6885c7d929f4f8e96d3890724c3b53f/roles/4eb30338d61e47808b51e050836643ea` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:20.074932 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 309/610] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Aug 4 21:03:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:20.075840 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb7da5a-66c2-4fa3-857a-e9a160850d05 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:20.079655 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7894835-46fe-49df-bce8-06b37990e224 None admin] RBAC: Authorizing `identity:create_grant(project_id=17dc3d39e2c048ca9070588f5717a578, user_id=f6885c7d929f4f8e96d3890724c3b53f, role_id=4eb30338d61e47808b51e050836643ea)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:20.081383 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7894835-46fe-49df-bce8-06b37990e224 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:20.163043 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 302/611] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Aug 4 21:03:19 2022] PUT /identity/v3/projects/17dc3d39e2c048ca9070588f5717a578/users/f6885c7d929f4f8e96d3890724c3b53f/roles/4eb30338d61e47808b51e050836643ea => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:03:20.164347 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7894835-46fe-49df-bce8-06b37990e224 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:20.192568 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb7da5a-66c2-4fa3-857a-e9a160850d05 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:20.249475 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533498ea-52de-43d3-aec7-cdbe23f4d4d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:20.250046 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533498ea-52de-43d3-aec7-cdbe23f4d4d1 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:20.250331 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533498ea-52de-43d3-aec7-cdbe23f4d4d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:20.250609 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533498ea-52de-43d3-aec7-cdbe23f4d4d1 None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:20.251346 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533498ea-52de-43d3-aec7-cdbe23f4d4d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:20.251662 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533498ea-52de-43d3-aec7-cdbe23f4d4d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:20.256093 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533498ea-52de-43d3-aec7-cdbe23f4d4d1 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:20.274397 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 310/612] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:03:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:20.278046 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c627f3-b676-41e2-8c65-4630d1fe0f5b None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:20.278377 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c627f3-b676-41e2-8c65-4630d1fe0f5b None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:20.278613 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c627f3-b676-41e2-8c65-4630d1fe0f5b None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:20.285122 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 311/613] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:03:20 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:03:20.286203 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c627f3-b676-41e2-8c65-4630d1fe0f5b None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:20.317928 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7894835-46fe-49df-bce8-06b37990e224 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:20.338889 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:20.354195 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:20.364519 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30f0a71-608c-4184-9b39-021e20319396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:20.365209 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30f0a71-608c-4184-9b39-021e20319396 None admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:20.365523 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30f0a71-608c-4184-9b39-021e20319396 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:20.365759 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30f0a71-608c-4184-9b39-021e20319396 None admin] PATH_INFO: `/v3/projects/17dc3d39e2c048ca9070588f5717a578/users/08fcb91ee54344b5bb3870c585e25fdc/roles/4eb30338d61e47808b51e050836643ea` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:20.384675 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30f0a71-608c-4184-9b39-021e20319396 None admin] RBAC: Authorizing `identity:create_grant(project_id=17dc3d39e2c048ca9070588f5717a578, user_id=08fcb91ee54344b5bb3870c585e25fdc, role_id=4eb30338d61e47808b51e050836643ea)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:20.386226 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30f0a71-608c-4184-9b39-021e20319396 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:20.411044 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c627f3-b676-41e2-8c65-4630d1fe0f5b None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:20.449163 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc9ea32-f219-48ce-aba1-75ebae82b206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:20.449854 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc9ea32-f219-48ce-aba1-75ebae82b206 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:20.450136 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc9ea32-f219-48ce-aba1-75ebae82b206 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:20.450409 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc9ea32-f219-48ce-aba1-75ebae82b206 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:20.457988 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc9ea32-f219-48ce-aba1-75ebae82b206 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:20.459490 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc9ea32-f219-48ce-aba1-75ebae82b206 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:20.464464 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 303/614] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Aug 4 21:03:20 2022] PUT /identity/v3/projects/17dc3d39e2c048ca9070588f5717a578/users/08fcb91ee54344b5bb3870c585e25fdc/roles/4eb30338d61e47808b51e050836643ea => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:03:20.465732 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b040b38-6d28-40fe-a5cc-5fba95fc7721 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:20.466018 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b040b38-6d28-40fe-a5cc-5fba95fc7721 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:20.471709 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b040b38-6d28-40fe-a5cc-5fba95fc7721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:20.478003 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-fbc9ea32-f219-48ce-aba1-75ebae82b206 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 04 21:03:20.478003 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:20.478003 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:20.478003 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:20.478003 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:20.478003 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:20.478003 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:20.478003 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:20.478003 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:20.478003 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:20.478003 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:20.478003 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:20.478003 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 21:03:20.478003 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 21:03:20.478003 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 21:03:20.478003 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 21:03:20.478971 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:20.478971 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:20.478971 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 21:03:20.478971 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:03:20.478971 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 21:03:20.478971 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:03:20.478971 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:20.478971 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:20.478971 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:20.478971 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:20.478971 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:20.478971 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:20.478971 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:20.478971 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:20.478971 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:20.478971 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:20.478971 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:20.478971 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:20.480623 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:20.480623 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:20.480623 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 21:03:20.480623 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 21:03:20.480623 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 21:03:20.480623 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 21:03:20.480623 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 21:03:20.480623 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 21:03:20.480623 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 04 21:03:20.480623 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:20.480623 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 312/615] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Aug 4 21:03:20 2022] GET /identity/v3/users/swift => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:20.484839 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc9ea32-f219-48ce-aba1-75ebae82b206 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:20.511197 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-1b040b38-6d28-40fe-a5cc-5fba95fc7721 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:03:20.513036 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-1b040b38-6d28-40fe-a5cc-5fba95fc7721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:03:20.628673 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 304/616] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:03:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:03:20.671037 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b040b38-6d28-40fe-a5cc-5fba95fc7721 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:20.687532 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc9ea32-f219-48ce-aba1-75ebae82b206 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:20.690575 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:20.725765 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2c4d41-43ce-46eb-aa91-53579e3dddc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:20.726741 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2c4d41-43ce-46eb-aa91-53579e3dddc8 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:20.727170 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2c4d41-43ce-46eb-aa91-53579e3dddc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:20.727614 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2c4d41-43ce-46eb-aa91-53579e3dddc8 None admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:20.728578 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2c4d41-43ce-46eb-aa91-53579e3dddc8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:20.729028 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2c4d41-43ce-46eb-aa91-53579e3dddc8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:20.731760 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2c4d41-43ce-46eb-aa91-53579e3dddc8 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:20.751095 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 313/617] 10.222.0.25 () {56 vars in 1135 bytes} [Thu Aug 4 21:03:20 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:20.759608 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2c4d41-43ce-46eb-aa91-53579e3dddc8 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:20.824510 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b040b38-6d28-40fe-a5cc-5fba95fc7721 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:20.855663 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:20.903249 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ae9fee-c3a0-4431-ad1c-fe012c45c83e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:20.903929 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae9fee-c3a0-4431-ad1c-fe012c45c83e None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:20.904211 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae9fee-c3a0-4431-ad1c-fe012c45c83e None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:20.904480 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae9fee-c3a0-4431-ad1c-fe012c45c83e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:20.931541 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ae9fee-c3a0-4431-ad1c-fe012c45c83e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:03:20.942849 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-f6ae9fee-c3a0-4431-ad1c-fe012c45c83e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:03:20.942849 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:20.942849 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:20.942849 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:20.942849 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:20.942849 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:20.942849 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:20.942849 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:20.942849 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:20.942849 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:20.942849 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:20.942849 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:20.942849 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:03:20.942849 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:03:20.942849 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:03:20.942849 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:03:20.943919 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:03:20.943919 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:03:20.943919 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:03:20.943919 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:03:20.943919 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:20.943919 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:20.943919 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:20.943919 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:20.943919 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:20.943919 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:20.943919 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:20.943919 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:20.943919 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:20.943919 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:20.943919 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:20.943919 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:20.944756 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:20.944756 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:20.944756 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:20.944756 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:20.944756 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:03:20.944756 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:03:20.944756 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:03:20.944756 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:03:20.944756 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:03:20.944756 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:03:20.944756 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:03:20.944756 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:20.947404 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 305/618] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Aug 4 21:03:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:03:20.952009 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ae9fee-c3a0-4431-ad1c-fe012c45c83e None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:20.979822 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2c4d41-43ce-46eb-aa91-53579e3dddc8 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:21.053030 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f744ba-b4e1-4e2f-af56-46a0ee2a2b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:21.053681 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f744ba-b4e1-4e2f-af56-46a0ee2a2b45 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:21.053963 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f744ba-b4e1-4e2f-af56-46a0ee2a2b45 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:21.054237 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f744ba-b4e1-4e2f-af56-46a0ee2a2b45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:21.080312 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f744ba-b4e1-4e2f-af56-46a0ee2a2b45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:21.081715 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f744ba-b4e1-4e2f-af56-46a0ee2a2b45 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:21.107348 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ae9fee-c3a0-4431-ad1c-fe012c45c83e None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:21.136152 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-39f744ba-b4e1-4e2f-af56-46a0ee2a2b45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:21.136152 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:21.136152 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:21.136152 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:21.136152 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:21.136152 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:21.136152 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:21.136152 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:21.136152 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:21.136152 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:21.136152 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:21.136152 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:21.136152 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:03:21.136152 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:03:21.136152 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:03:21.137117 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:03:21.137117 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:21.137117 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:21.137117 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:21.137117 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:21.137117 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:21.137117 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:21.137117 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:21.137117 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:21.137117 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:21.137117 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:21.137117 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:21.137117 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:21.137117 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:21.137117 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:21.137117 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:21.137117 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:21.138684 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:03:21.138684 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:03:21.138684 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:21.138684 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:21.138684 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 314/619] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:03:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:21.143014 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f744ba-b4e1-4e2f-af56-46a0ee2a2b45 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:21.152064 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fd42a1-3796-431a-b019-191d1b95396d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:21.152698 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fd42a1-3796-431a-b019-191d1b95396d None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:21.152981 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fd42a1-3796-431a-b019-191d1b95396d None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:21.153253 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fd42a1-3796-431a-b019-191d1b95396d None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:21.153975 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fd42a1-3796-431a-b019-191d1b95396d None admin] RBAC: Adding query filter params (name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:21.154280 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fd42a1-3796-431a-b019-191d1b95396d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:21.158049 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fd42a1-3796-431a-b019-191d1b95396d None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:21.174911 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:21.176726 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 306/620] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Aug 4 21:03:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:03:21.182049 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fd42a1-3796-431a-b019-191d1b95396d None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:21.269330 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f744ba-b4e1-4e2f-af56-46a0ee2a2b45 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:21.281614 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:21.325833 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2047c4fb-7121-4cdb-9712-b1683a65f0c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:21.326467 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2047c4fb-7121-4cdb-9712-b1683a65f0c0 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:21.326745 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2047c4fb-7121-4cdb-9712-b1683a65f0c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:21.327017 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2047c4fb-7121-4cdb-9712-b1683a65f0c0 None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:21.327743 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2047c4fb-7121-4cdb-9712-b1683a65f0c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:21.328048 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2047c4fb-7121-4cdb-9712-b1683a65f0c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:21.329426 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2047c4fb-7121-4cdb-9712-b1683a65f0c0 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:21.359876 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 315/621] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:03:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:21.363446 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fd42a1-3796-431a-b019-191d1b95396d None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:21.365250 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2047c4fb-7121-4cdb-9712-b1683a65f0c0 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:21.400813 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8563d1b-48d1-490b-a6de-06e17aac02d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:21.401520 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8563d1b-48d1-490b-a6de-06e17aac02d4 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:21.401810 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8563d1b-48d1-490b-a6de-06e17aac02d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:21.402082 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8563d1b-48d1-490b-a6de-06e17aac02d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:21.413904 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8563d1b-48d1-490b-a6de-06e17aac02d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:21.415776 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8563d1b-48d1-490b-a6de-06e17aac02d4 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:21.434862 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-a8563d1b-48d1-490b-a6de-06e17aac02d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:21.434862 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:21.434862 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:21.434862 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:21.434862 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:21.434862 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:21.434862 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:21.434862 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:21.434862 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:21.434862 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:21.434862 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:21.434862 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:21.434862 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:03:21.434862 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:03:21.434862 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:03:21.435827 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:03:21.435827 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:21.435827 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:21.435827 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:21.435827 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:21.435827 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:21.435827 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:21.435827 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:21.435827 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:21.435827 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:21.435827 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:21.435827 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:21.435827 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:21.435827 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:21.435827 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:21.435827 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:21.435827 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:21.437319 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:03:21.437319 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:03:21.437319 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:21.437319 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:21.437319 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 307/622] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:03:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:03:21.439604 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b910f16-de66-49f5-bee3-adf2584b5cc6 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:21.440494 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b910f16-de66-49f5-bee3-adf2584b5cc6 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:21.440772 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b910f16-de66-49f5-bee3-adf2584b5cc6 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:21.442608 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 308/623] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:03:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:03:21.443424 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b910f16-de66-49f5-bee3-adf2584b5cc6 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:21.464480 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:21.490560 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2047c4fb-7121-4cdb-9712-b1683a65f0c0 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:21.525723 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c308d0-4a84-4069-91be-2804fd6cbdc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:21.526362 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c308d0-4a84-4069-91be-2804fd6cbdc6 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:21.526640 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c308d0-4a84-4069-91be-2804fd6cbdc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:21.526910 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c308d0-4a84-4069-91be-2804fd6cbdc6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:21.534076 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c308d0-4a84-4069-91be-2804fd6cbdc6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:21.535530 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c308d0-4a84-4069-91be-2804fd6cbdc6 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:21.547540 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-04c308d0-4a84-4069-91be-2804fd6cbdc6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:03:21.547540 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:21.547540 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:21.547540 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:21.547540 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:21.547540 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:21.547540 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:21.547540 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:21.547540 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:21.547540 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:21.547540 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:21.547540 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:21.547540 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 21:03:21.547540 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 21:03:21.547540 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 21:03:21.548444 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 21:03:21.548444 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:21.548444 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:21.548444 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:21.548444 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:21.548444 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:21.548444 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:21.548444 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:21.548444 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:21.548444 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:21.548444 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:21.548444 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:21.548444 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:21.548444 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:21.548444 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:21.548444 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:21.548444 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:21.549865 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 21:03:21.549865 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 21:03:21.549865 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 21:03:21.549865 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 21:03:21.549865 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 21:03:21.549865 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 21:03:21.549865 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:03:21.549865 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:21.553846 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 316/624] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Aug 4 21:03:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:21.555262 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013886d0-d181-4647-9ca0-5807ce78c779 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:21.555964 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:21.556751 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013886d0-d181-4647-9ca0-5807ce78c779 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:21.556926 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b910f16-de66-49f5-bee3-adf2584b5cc6 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:21.557105 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013886d0-d181-4647-9ca0-5807ce78c779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:21.574764 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-013886d0-d181-4647-9ca0-5807ce78c779 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:03:21.576043 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-013886d0-d181-4647-9ca0-5807ce78c779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:03:21.594080 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a818a9f5-0dfb-4133-aa68-825c282f3cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:21.594804 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a818a9f5-0dfb-4133-aa68-825c282f3cbb None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:21.595137 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a818a9f5-0dfb-4133-aa68-825c282f3cbb None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:21.595442 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a818a9f5-0dfb-4133-aa68-825c282f3cbb None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:21.596386 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a818a9f5-0dfb-4133-aa68-825c282f3cbb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:21.596693 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a818a9f5-0dfb-4133-aa68-825c282f3cbb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:21.598133 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a818a9f5-0dfb-4133-aa68-825c282f3cbb None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:21.611001 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 309/625] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:03:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:21.612417 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a818a9f5-0dfb-4133-aa68-825c282f3cbb None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:21.655499 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 317/626] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:03:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:03:21.656360 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013886d0-d181-4647-9ca0-5807ce78c779 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:21.720032 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a818a9f5-0dfb-4133-aa68-825c282f3cbb None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:21.735626 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:21.757348 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff3ee7d-5c58-43b5-82f7-e58052adf7cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:21.758005 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff3ee7d-5c58-43b5-82f7-e58052adf7cd None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:21.758289 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff3ee7d-5c58-43b5-82f7-e58052adf7cd None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:21.758562 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff3ee7d-5c58-43b5-82f7-e58052adf7cd None admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:21.759388 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff3ee7d-5c58-43b5-82f7-e58052adf7cd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:21.759699 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff3ee7d-5c58-43b5-82f7-e58052adf7cd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:21.761145 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff3ee7d-5c58-43b5-82f7-e58052adf7cd None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:21.774173 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 310/627] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Aug 4 21:03:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:21.775083 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff3ee7d-5c58-43b5-82f7-e58052adf7cd None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:21.776725 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013886d0-d181-4647-9ca0-5807ce78c779 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:21.790647 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:21.814330 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e61988d-77d2-4266-9845-2a133d0a0131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:21.815030 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e61988d-77d2-4266-9845-2a133d0a0131 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:21.815372 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e61988d-77d2-4266-9845-2a133d0a0131 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:21.815607 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e61988d-77d2-4266-9845-2a133d0a0131 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:21.824445 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e61988d-77d2-4266-9845-2a133d0a0131 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:21.826284 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e61988d-77d2-4266-9845-2a133d0a0131 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:21.839520 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-6e61988d-77d2-4266-9845-2a133d0a0131 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 21:03:21.839520 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:21.839520 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:21.839520 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:21.839520 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:21.839520 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:21.839520 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:21.839520 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:21.839520 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:21.839520 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:21.839520 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:21.839520 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:21.839520 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 21:03:21.839520 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 21:03:21.839520 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 21:03:21.839520 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 21:03:21.841908 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:21.841908 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:21.841908 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 21:03:21.841908 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:03:21.841908 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 21:03:21.841908 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:03:21.841908 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:21.841908 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:21.841908 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:21.841908 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:21.841908 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:21.841908 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:21.841908 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:21.841908 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:21.841908 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:21.841908 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:21.841908 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:21.841908 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:21.844274 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:21.844274 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:21.844274 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 21:03:21.844274 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 21:03:21.844274 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 21:03:21.844274 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 21:03:21.844274 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 21:03:21.844274 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 21:03:21.844274 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 21:03:21.844274 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:21.844274 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 318/628] 10.222.0.25 () {56 vars in 1093 bytes} [Thu Aug 4 21:03:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:21.844274 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e61988d-77d2-4266-9845-2a133d0a0131 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:21.892774 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff3ee7d-5c58-43b5-82f7-e58052adf7cd None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:21.936020 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29036fd-cfac-4aae-a738-01c62de5507f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:21.937110 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29036fd-cfac-4aae-a738-01c62de5507f None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:21.937421 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29036fd-cfac-4aae-a738-01c62de5507f None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:21.937704 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29036fd-cfac-4aae-a738-01c62de5507f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:21.949517 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e61988d-77d2-4266-9845-2a133d0a0131 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:21.950518 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29036fd-cfac-4aae-a738-01c62de5507f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:03:21.956239 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-d29036fd-cfac-4aae-a738-01c62de5507f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:03:21.956239 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:21.956239 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:21.956239 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:21.956239 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:21.956239 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:21.956239 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:21.956239 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:21.956239 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:21.956239 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:21.956239 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:21.956239 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:21.956239 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:03:21.956239 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:03:21.956239 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:03:21.956239 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:03:21.958353 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:03:21.958353 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:03:21.958353 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:03:21.958353 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:03:21.958353 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:21.958353 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:21.958353 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:21.958353 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:21.958353 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:21.958353 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:21.958353 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:21.958353 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:21.958353 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:21.958353 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:21.958353 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:21.958353 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:21.960446 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:21.960446 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:21.960446 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:21.960446 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:21.960446 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:03:21.960446 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:03:21.960446 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:03:21.960446 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:03:21.960446 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:03:21.960446 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:03:21.960446 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:03:21.960446 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:21.960446 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 311/629] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Aug 4 21:03:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:21.960446 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29036fd-cfac-4aae-a738-01c62de5507f None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:21.983554 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:21.996965 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f06adf-27da-4302-a032-62b7a5dbccf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:21.998076 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f06adf-27da-4302-a032-62b7a5dbccf4 None admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:21.998600 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f06adf-27da-4302-a032-62b7a5dbccf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:21.998891 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f06adf-27da-4302-a032-62b7a5dbccf4 None admin] PATH_INFO: `/v3/projects/17dc3d39e2c048ca9070588f5717a578/users/58935367c0ad445f8477cdac0df04d0a/roles/4eb30338d61e47808b51e050836643ea` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:22.017423 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f06adf-27da-4302-a032-62b7a5dbccf4 None admin] RBAC: Authorizing `identity:create_grant(project_id=17dc3d39e2c048ca9070588f5717a578, user_id=58935367c0ad445f8477cdac0df04d0a, role_id=4eb30338d61e47808b51e050836643ea)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:22.020938 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f06adf-27da-4302-a032-62b7a5dbccf4 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:22.046943 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:22.082651 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29036fd-cfac-4aae-a738-01c62de5507f None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:22.107966 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 319/630] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Aug 4 21:03:21 2022] PUT /identity/v3/projects/17dc3d39e2c048ca9070588f5717a578/users/58935367c0ad445f8477cdac0df04d0a/roles/4eb30338d61e47808b51e050836643ea => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:03:22.108897 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f06adf-27da-4302-a032-62b7a5dbccf4 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:22.131975 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b937b6d8-a479-4468-a714-f3503cc5f0c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:22.133076 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b937b6d8-a479-4468-a714-f3503cc5f0c9 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:22.133572 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b937b6d8-a479-4468-a714-f3503cc5f0c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:22.134076 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b937b6d8-a479-4468-a714-f3503cc5f0c9 None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:22.135111 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b937b6d8-a479-4468-a714-f3503cc5f0c9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:22.135725 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b937b6d8-a479-4468-a714-f3503cc5f0c9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:22.137734 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b937b6d8-a479-4468-a714-f3503cc5f0c9 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:22.168496 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 312/631] 10.222.0.25 () {56 vars in 1133 bytes} [Thu Aug 4 21:03:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:22.174223 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b937b6d8-a479-4468-a714-f3503cc5f0c9 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:22.270329 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f06adf-27da-4302-a032-62b7a5dbccf4 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:22.309846 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c3c00f-1fe9-491a-8d87-cb4e6599690d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:22.310535 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c3c00f-1fe9-491a-8d87-cb4e6599690d None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:22.310808 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c3c00f-1fe9-491a-8d87-cb4e6599690d None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:22.311072 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c3c00f-1fe9-491a-8d87-cb4e6599690d None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:22.312056 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c3c00f-1fe9-491a-8d87-cb4e6599690d None admin] RBAC: Adding query filter params (name=service) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:22.312364 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c3c00f-1fe9-491a-8d87-cb4e6599690d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:22.313784 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c3c00f-1fe9-491a-8d87-cb4e6599690d None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:22.323800 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 320/632] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Aug 4 21:03:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:22.329028 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c3c00f-1fe9-491a-8d87-cb4e6599690d None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:22.369950 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:22.390709 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b937b6d8-a479-4468-a714-f3503cc5f0c9 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:22.424304 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:22.439032 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c3c00f-1fe9-491a-8d87-cb4e6599690d None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:22.439563 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480ced06-d507-42e2-8573-c361978016eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:22.440226 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480ced06-d507-42e2-8573-c361978016eb None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:22.440508 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480ced06-d507-42e2-8573-c361978016eb None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:22.440951 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480ced06-d507-42e2-8573-c361978016eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:22.519720 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480ced06-d507-42e2-8573-c361978016eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:22.521231 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480ced06-d507-42e2-8573-c361978016eb None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:22.620715 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb47e974-50a8-40d7-a733-17fba847ab71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:22.621385 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb47e974-50a8-40d7-a733-17fba847ab71 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:22.621671 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb47e974-50a8-40d7-a733-17fba847ab71 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:22.621926 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb47e974-50a8-40d7-a733-17fba847ab71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:22.637126 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb47e974-50a8-40d7-a733-17fba847ab71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:22.638671 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb47e974-50a8-40d7-a733-17fba847ab71 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:22.639940 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 313/633] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Aug 4 21:03:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 04 21:03:22.641818 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91615214-0f0b-4792-a02a-7cf0d977f8a8 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:22.642316 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91615214-0f0b-4792-a02a-7cf0d977f8a8 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:22.643357 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91615214-0f0b-4792-a02a-7cf0d977f8a8 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:22.649231 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 314/634] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:03:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:03:22.650209 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91615214-0f0b-4792-a02a-7cf0d977f8a8 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:22.656311 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-eb47e974-50a8-40d7-a733-17fba847ab71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:22.656311 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:22.656311 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:22.656311 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:22.656311 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:22.656311 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:22.656311 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:22.656311 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:22.656311 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:22.656311 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:22.656311 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:22.656311 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:22.656311 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:03:22.656311 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:03:22.656311 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:03:22.657224 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:03:22.657224 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:22.657224 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:22.657224 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:22.657224 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:22.657224 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:22.657224 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:22.657224 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:22.657224 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:22.657224 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:22.657224 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:22.657224 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:22.657224 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:22.657224 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:22.657224 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:22.657224 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:22.657224 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:22.658153 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:03:22.658153 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:03:22.658153 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:22.658153 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:22.658153 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 321/635] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:03:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:03:22.659437 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d171f765-73ee-4c8d-a79d-215b883e3f28 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:22.659722 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d171f765-73ee-4c8d-a79d-215b883e3f28 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:22.659982 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d171f765-73ee-4c8d-a79d-215b883e3f28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:22.681270 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:22.685792 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-d171f765-73ee-4c8d-a79d-215b883e3f28 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:03:22.687234 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-d171f765-73ee-4c8d-a79d-215b883e3f28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:03:22.766052 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91615214-0f0b-4792-a02a-7cf0d977f8a8 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:22.787343 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 322/636] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:03:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:03:22.787343 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d171f765-73ee-4c8d-a79d-215b883e3f28 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:22.801524 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:22.826500 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4288a6d3-407d-4bcb-a127-5fde87786d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:22.827155 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4288a6d3-407d-4bcb-a127-5fde87786d69 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:22.827479 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4288a6d3-407d-4bcb-a127-5fde87786d69 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:22.827756 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4288a6d3-407d-4bcb-a127-5fde87786d69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:22.838831 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4288a6d3-407d-4bcb-a127-5fde87786d69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:22.840430 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4288a6d3-407d-4bcb-a127-5fde87786d69 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:22.857426 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-4288a6d3-407d-4bcb-a127-5fde87786d69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:22.857426 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:22.857426 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:22.857426 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:22.857426 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:22.857426 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:22.857426 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:22.857426 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:22.857426 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:22.857426 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:22.857426 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:22.857426 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:22.857426 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:03:22.857426 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:03:22.857426 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:03:22.858372 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:03:22.858372 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:22.858372 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:22.858372 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:22.858372 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:22.858372 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:22.858372 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:22.858372 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:22.858372 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:22.858372 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:22.858372 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:22.858372 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:22.858372 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:22.858372 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:22.858372 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:22.858372 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:22.858372 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:22.864689 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:03:22.864689 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:03:22.864689 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:22.864689 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:22.864689 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 315/637] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:03:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:22.864689 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4288a6d3-407d-4bcb-a127-5fde87786d69 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:22.888588 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d171f765-73ee-4c8d-a79d-215b883e3f28 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:22.926347 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c83489-74a2-4244-9bea-3e227e848ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:22.927310 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c83489-74a2-4244-9bea-3e227e848ffe None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:22.927622 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c83489-74a2-4244-9bea-3e227e848ffe None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:22.927901 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c83489-74a2-4244-9bea-3e227e848ffe None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:22.928598 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c83489-74a2-4244-9bea-3e227e848ffe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:22.928909 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c83489-74a2-4244-9bea-3e227e848ffe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:22.930289 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c83489-74a2-4244-9bea-3e227e848ffe None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:22.942327 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 323/638] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:03:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:22.943493 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b09012e-12ba-4429-8e9c-e8a7379b4009 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:22.943783 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b09012e-12ba-4429-8e9c-e8a7379b4009 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:22.944056 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b09012e-12ba-4429-8e9c-e8a7379b4009 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:22.947641 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 324/639] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:03:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:03:22.948425 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b09012e-12ba-4429-8e9c-e8a7379b4009 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:22.950078 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:22.978346 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4288a6d3-407d-4bcb-a127-5fde87786d69 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:23.012737 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758fe216-d1c0-4e34-a6a2-616826553d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:23.013381 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758fe216-d1c0-4e34-a6a2-616826553d82 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:23.013674 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758fe216-d1c0-4e34-a6a2-616826553d82 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:23.013944 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758fe216-d1c0-4e34-a6a2-616826553d82 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:23.025237 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758fe216-d1c0-4e34-a6a2-616826553d82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:03:23.030624 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-758fe216-d1c0-4e34-a6a2-616826553d82 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:03:23.030624 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:23.030624 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:23.030624 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:23.030624 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:23.030624 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:23.030624 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:23.030624 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:23.030624 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:23.030624 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:23.030624 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:23.030624 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:23.030624 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:03:23.030624 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:03:23.030624 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:03:23.030624 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:03:23.041790 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:03:23.041790 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:03:23.041790 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:03:23.041790 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:03:23.041790 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:23.041790 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:23.041790 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:23.041790 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:23.041790 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:23.041790 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:23.041790 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:23.041790 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:23.041790 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:23.041790 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:23.041790 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:23.041790 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:23.042758 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:23.042758 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:23.042758 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:23.042758 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:23.042758 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:03:23.042758 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:03:23.042758 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:03:23.042758 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:03:23.042758 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:03:23.042758 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:03:23.042758 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:03:23.042758 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:23.042758 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 316/640] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Aug 4 21:03:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:23.042758 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758fe216-d1c0-4e34-a6a2-616826553d82 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:23.045283 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:23.067294 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b09012e-12ba-4429-8e9c-e8a7379b4009 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:23.102051 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a5031d-2ef4-455c-80ff-53dea4b5f49f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:23.102924 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a5031d-2ef4-455c-80ff-53dea4b5f49f None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:23.103449 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a5031d-2ef4-455c-80ff-53dea4b5f49f None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:23.104008 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a5031d-2ef4-455c-80ff-53dea4b5f49f None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:23.104866 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a5031d-2ef4-455c-80ff-53dea4b5f49f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:23.105449 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a5031d-2ef4-455c-80ff-53dea4b5f49f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:23.107017 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a5031d-2ef4-455c-80ff-53dea4b5f49f None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:23.119170 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 325/641] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:03:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:23.122297 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32515dc6-c896-4844-9a34-46b565343ef6 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:23.122635 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32515dc6-c896-4844-9a34-46b565343ef6 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:23.122965 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32515dc6-c896-4844-9a34-46b565343ef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:23.139871 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758fe216-d1c0-4e34-a6a2-616826553d82 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:23.141006 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-32515dc6-c896-4844-9a34-46b565343ef6 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:03:23.142311 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-32515dc6-c896-4844-9a34-46b565343ef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:03:23.178252 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc64ddb8-d3a0-4612-a1b7-3906b6321e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:23.179114 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc64ddb8-d3a0-4612-a1b7-3906b6321e8d None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:23.179503 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc64ddb8-d3a0-4612-a1b7-3906b6321e8d None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:23.179843 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc64ddb8-d3a0-4612-a1b7-3906b6321e8d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:23.182689 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:23.188200 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc64ddb8-d3a0-4612-a1b7-3906b6321e8d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:23.190392 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc64ddb8-d3a0-4612-a1b7-3906b6321e8d None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:23.201569 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-fc64ddb8-d3a0-4612-a1b7-3906b6321e8d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 21:03:23.201569 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:23.201569 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:23.201569 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:23.201569 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:23.201569 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:23.201569 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:23.201569 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:23.201569 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:23.201569 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:23.201569 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:23.201569 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:23.201569 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 21:03:23.201569 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 21:03:23.201569 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 21:03:23.201569 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 21:03:23.202739 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:23.202739 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:23.202739 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 21:03:23.202739 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:03:23.202739 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 21:03:23.202739 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:03:23.202739 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:23.202739 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:23.202739 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:23.202739 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:23.202739 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:23.202739 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:23.202739 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:23.202739 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:23.202739 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:23.202739 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:23.202739 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:23.202739 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:23.203920 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:23.203920 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:23.203920 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 21:03:23.203920 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 21:03:23.203920 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 21:03:23.203920 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 21:03:23.203920 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 21:03:23.203920 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 21:03:23.203920 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 21:03:23.203920 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:23.210878 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 317/642] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Aug 4 21:03:23 2022] GET /identity/v3/users/cinder => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:23.210878 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc64ddb8-d3a0-4612-a1b7-3906b6321e8d None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:23.240291 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 326/643] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:03:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:03:23.241355 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32515dc6-c896-4844-9a34-46b565343ef6 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:23.321645 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:23.323663 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc64ddb8-d3a0-4612-a1b7-3906b6321e8d None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:23.356641 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32515dc6-c896-4844-9a34-46b565343ef6 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:23.358134 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815f33f0-8fa2-4c03-8099-03aa076662c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:23.359554 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815f33f0-8fa2-4c03-8099-03aa076662c8 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:23.359867 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815f33f0-8fa2-4c03-8099-03aa076662c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:23.360203 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815f33f0-8fa2-4c03-8099-03aa076662c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:23.360980 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815f33f0-8fa2-4c03-8099-03aa076662c8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:23.361344 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815f33f0-8fa2-4c03-8099-03aa076662c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:23.363224 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815f33f0-8fa2-4c03-8099-03aa076662c8 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:23.375359 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 318/644] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Aug 4 21:03:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:23.375359 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815f33f0-8fa2-4c03-8099-03aa076662c8 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:23.393576 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8761c5c1-4676-4a80-a3bb-c1c82d604b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:23.394305 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8761c5c1-4676-4a80-a3bb-c1c82d604b09 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:23.394588 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8761c5c1-4676-4a80-a3bb-c1c82d604b09 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:23.394852 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8761c5c1-4676-4a80-a3bb-c1c82d604b09 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:23.402581 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8761c5c1-4676-4a80-a3bb-c1c82d604b09 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:23.404104 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8761c5c1-4676-4a80-a3bb-c1c82d604b09 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:23.418113 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-8761c5c1-4676-4a80-a3bb-c1c82d604b09 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:03:23.418113 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:23.418113 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:23.418113 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:23.418113 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:23.418113 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:23.418113 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:23.418113 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:23.418113 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:23.418113 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:23.418113 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:23.418113 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:23.418113 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 21:03:23.418113 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 21:03:23.418113 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 21:03:23.419062 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 21:03:23.419062 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:23.419062 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:23.419062 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:23.419062 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:23.419062 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:23.419062 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:23.419062 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:23.419062 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:23.419062 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:23.419062 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:23.419062 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:23.419062 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:23.419062 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:23.419062 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:23.419062 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:23.419062 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:23.420389 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 21:03:23.420389 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 21:03:23.420389 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 21:03:23.420389 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 21:03:23.420389 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 21:03:23.420389 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 21:03:23.420389 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:03:23.420389 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:23.420389 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 327/645] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Aug 4 21:03:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:23.420389 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8761c5c1-4676-4a80-a3bb-c1c82d604b09 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:23.441738 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:23.483612 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815f33f0-8fa2-4c03-8099-03aa076662c8 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:23.518522 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79412d1b-7a67-4ded-9bee-a6ced9686727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:23.519158 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79412d1b-7a67-4ded-9bee-a6ced9686727 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:23.519467 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79412d1b-7a67-4ded-9bee-a6ced9686727 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:23.519750 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79412d1b-7a67-4ded-9bee-a6ced9686727 None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:23.520540 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79412d1b-7a67-4ded-9bee-a6ced9686727 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:23.520847 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79412d1b-7a67-4ded-9bee-a6ced9686727 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:23.522301 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79412d1b-7a67-4ded-9bee-a6ced9686727 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:23.529097 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8761c5c1-4676-4a80-a3bb-c1c82d604b09 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:23.539900 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 319/646] 10.222.0.25 () {56 vars in 1137 bytes} [Thu Aug 4 21:03:23 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:23.540708 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79412d1b-7a67-4ded-9bee-a6ced9686727 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:23.561841 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:23.571188 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cfb6e8-4790-418b-b05e-d49704026b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:23.572040 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cfb6e8-4790-418b-b05e-d49704026b3c None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:23.572334 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cfb6e8-4790-418b-b05e-d49704026b3c None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:23.572609 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cfb6e8-4790-418b-b05e-d49704026b3c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:23.583534 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cfb6e8-4790-418b-b05e-d49704026b3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:03:23.588450 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-00cfb6e8-4790-418b-b05e-d49704026b3c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:03:23.588450 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:23.588450 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:23.588450 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:23.588450 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:23.588450 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:23.588450 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:23.588450 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:23.588450 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:23.588450 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:23.588450 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:23.588450 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:23.588450 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:03:23.588450 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:03:23.588450 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:03:23.588450 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:03:23.589434 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:03:23.589434 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:03:23.589434 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:03:23.589434 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:03:23.589434 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:23.589434 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:23.589434 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:23.589434 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:23.589434 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:23.589434 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:23.589434 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:23.589434 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:23.589434 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:23.589434 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:23.589434 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:23.589434 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:23.590274 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:23.590274 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:23.590274 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:23.590274 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:23.590274 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:03:23.590274 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:03:23.590274 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:03:23.590274 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:03:23.590274 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:03:23.590274 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:03:23.590274 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:03:23.590274 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:23.590274 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 328/647] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Aug 4 21:03:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:23.590966 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cfb6e8-4790-418b-b05e-d49704026b3c None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:23.661780 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79412d1b-7a67-4ded-9bee-a6ced9686727 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:23.678411 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:23.695941 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecef3ef-0a46-4e48-b12a-a18d78d36437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:23.696579 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecef3ef-0a46-4e48-b12a-a18d78d36437 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:23.696865 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecef3ef-0a46-4e48-b12a-a18d78d36437 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:23.697138 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecef3ef-0a46-4e48-b12a-a18d78d36437 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:23.698734 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cfb6e8-4790-418b-b05e-d49704026b3c None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:23.710043 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecef3ef-0a46-4e48-b12a-a18d78d36437 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:23.711760 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecef3ef-0a46-4e48-b12a-a18d78d36437 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:23.729059 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-9ecef3ef-0a46-4e48-b12a-a18d78d36437 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:23.729059 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:23.729059 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:23.729059 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:23.729059 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:23.729059 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:23.729059 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:23.729059 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:23.729059 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:23.729059 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:23.729059 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:23.729059 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:23.729059 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:03:23.729059 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:03:23.729059 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:03:23.729976 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:03:23.729976 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:23.729976 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:23.729976 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:23.729976 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:23.729976 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:23.729976 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:23.729976 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:23.729976 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:23.729976 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:23.729976 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:23.729976 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:23.729976 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:23.729976 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:23.729976 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:23.729976 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:23.729976 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:23.730844 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:03:23.730844 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:03:23.730844 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:23.730844 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:23.730844 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 320/648] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:03:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:23.731127 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecef3ef-0a46-4e48-b12a-a18d78d36437 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:23.739965 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e05f880-fb49-435f-b996-9e534fd7e21f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:23.740613 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e05f880-fb49-435f-b996-9e534fd7e21f None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:23.740962 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e05f880-fb49-435f-b996-9e534fd7e21f None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:23.741697 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e05f880-fb49-435f-b996-9e534fd7e21f None admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:23.742528 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e05f880-fb49-435f-b996-9e534fd7e21f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:23.743235 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e05f880-fb49-435f-b996-9e534fd7e21f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:23.745122 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e05f880-fb49-435f-b996-9e534fd7e21f None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:23.757915 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 329/649] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Aug 4 21:03:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:23.758816 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e05f880-fb49-435f-b996-9e534fd7e21f None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:23.811909 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:23.844189 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecef3ef-0a46-4e48-b12a-a18d78d36437 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:23.869187 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e05f880-fb49-435f-b996-9e534fd7e21f None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:23.882807 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b25339-bbf7-4be4-9e43-30819271e57d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:23.883711 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b25339-bbf7-4be4-9e43-30819271e57d None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:23.884154 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b25339-bbf7-4be4-9e43-30819271e57d None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:23.884431 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b25339-bbf7-4be4-9e43-30819271e57d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:23.905512 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01353b6f-3333-48de-afd6-4cca6b52a318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:23.906298 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01353b6f-3333-48de-afd6-4cca6b52a318 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:23.906923 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01353b6f-3333-48de-afd6-4cca6b52a318 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:23.907293 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01353b6f-3333-48de-afd6-4cca6b52a318 None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:23.909185 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01353b6f-3333-48de-afd6-4cca6b52a318 None admin] RBAC: Adding query filter params (name=service) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:23.909642 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01353b6f-3333-48de-afd6-4cca6b52a318 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:23.911237 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01353b6f-3333-48de-afd6-4cca6b52a318 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:23.925506 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 330/650] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Aug 4 21:03:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:23.925506 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01353b6f-3333-48de-afd6-4cca6b52a318 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:24.038862 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:24.039332 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01353b6f-3333-48de-afd6-4cca6b52a318 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:24.052148 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:24.075131 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b25339-bbf7-4be4-9e43-30819271e57d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:24.076906 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b25339-bbf7-4be4-9e43-30819271e57d None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:24.077735 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41c03b2-aa2f-4a4b-a34e-84de158a010a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:24.078357 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41c03b2-aa2f-4a4b-a34e-84de158a010a None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:24.078641 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41c03b2-aa2f-4a4b-a34e-84de158a010a None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:24.078922 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41c03b2-aa2f-4a4b-a34e-84de158a010a None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:24.079645 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41c03b2-aa2f-4a4b-a34e-84de158a010a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:24.079953 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41c03b2-aa2f-4a4b-a34e-84de158a010a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:24.081373 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41c03b2-aa2f-4a4b-a34e-84de158a010a None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:24.092935 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 331/651] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:03:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:24.093585 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41c03b2-aa2f-4a4b-a34e-84de158a010a None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:24.195288 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 321/652] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Aug 4 21:03:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 21:03:24.195288 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b25339-bbf7-4be4-9e43-30819271e57d None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:24.204001 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41c03b2-aa2f-4a4b-a34e-84de158a010a None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:24.247635 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ebb5d3-4f19-4e64-94f1-bd14eda50f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:24.248378 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ebb5d3-4f19-4e64-94f1-bd14eda50f5d None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:24.248665 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ebb5d3-4f19-4e64-94f1-bd14eda50f5d None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:24.248938 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ebb5d3-4f19-4e64-94f1-bd14eda50f5d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:24.249719 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ebb5d3-4f19-4e64-94f1-bd14eda50f5d None admin] RBAC: Adding query filter params (role.id=4eb30338d61e47808b51e050836643ea, scope.project.id=17dc3d39e2c048ca9070588f5717a578, user.id=6a2ce1df87fa47e5be301916bf3245fb) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:24.250027 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ebb5d3-4f19-4e64-94f1-bd14eda50f5d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:24.251484 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ebb5d3-4f19-4e64-94f1-bd14eda50f5d None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:24.267505 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 332/653] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Aug 4 21:03:24 2022] GET /identity/v3/role_assignments?user.id=6a2ce1df87fa47e5be301916bf3245fb&scope.project.id=17dc3d39e2c048ca9070588f5717a578&role.id=4eb30338d61e47808b51e050836643ea => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:24.267505 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ebb5d3-4f19-4e64-94f1-bd14eda50f5d None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:24.273451 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:24.309945 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:24.313901 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b25339-bbf7-4be4-9e43-30819271e57d None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:24.354291 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e34a01-8603-4fa8-bb3b-e031c2d753fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:24.355064 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e34a01-8603-4fa8-bb3b-e031c2d753fe None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:24.355381 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e34a01-8603-4fa8-bb3b-e031c2d753fe None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:24.355661 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e34a01-8603-4fa8-bb3b-e031c2d753fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:24.367093 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e34a01-8603-4fa8-bb3b-e031c2d753fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:24.368621 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e34a01-8603-4fa8-bb3b-e031c2d753fe None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:24.384065 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-a2e34a01-8603-4fa8-bb3b-e031c2d753fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:24.384065 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:24.384065 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:24.384065 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:24.384065 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:24.384065 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:24.384065 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:24.384065 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:24.384065 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:24.384065 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:24.384065 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:24.384065 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:24.384065 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:03:24.384065 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:03:24.384065 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:03:24.385071 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:03:24.385071 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:24.385071 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:24.385071 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:24.385071 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:24.385071 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:24.385071 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:24.385071 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:24.385071 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:24.385071 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:24.385071 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:24.385071 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:24.385071 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:24.385071 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:24.385071 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:24.385071 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:24.385071 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:24.386056 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:03:24.386056 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:03:24.386056 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:24.386056 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:24.386280 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ebb5d3-4f19-4e64-94f1-bd14eda50f5d None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:24.386363 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 322/654] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:03:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:24.386688 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e34a01-8603-4fa8-bb3b-e031c2d753fe None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:24.430059 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fc2ca8-f42b-42b7-a2c3-53a1845a8331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:24.430789 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc2ca8-f42b-42b7-a2c3-53a1845a8331 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:24.431072 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc2ca8-f42b-42b7-a2c3-53a1845a8331 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:24.431417 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc2ca8-f42b-42b7-a2c3-53a1845a8331 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:24.438817 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fc2ca8-f42b-42b7-a2c3-53a1845a8331 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:24.440729 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fc2ca8-f42b-42b7-a2c3-53a1845a8331 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:24.453518 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-75fc2ca8-f42b-42b7-a2c3-53a1845a8331 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 04 21:03:24.453518 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:24.453518 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:24.453518 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:24.453518 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:24.453518 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:24.453518 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:24.453518 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:24.453518 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:24.453518 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:24.453518 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:24.453518 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:24.453518 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 21:03:24.453518 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 21:03:24.453518 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 21:03:24.453518 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 21:03:24.454495 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:24.454495 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:24.454495 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 21:03:24.454495 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:03:24.454495 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 21:03:24.454495 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:03:24.454495 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:24.454495 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:24.454495 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:24.454495 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:24.454495 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:24.454495 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:24.454495 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:24.454495 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:24.454495 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:24.454495 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:24.454495 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:24.454495 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:24.456108 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:24.456108 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:24.456108 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 21:03:24.456108 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 21:03:24.456108 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 21:03:24.456108 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 21:03:24.456108 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 21:03:24.456108 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 21:03:24.456108 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 04 21:03:24.456108 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:24.456108 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 333/655] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Aug 4 21:03:24 2022] GET /identity/v3/users/glance => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:24.456714 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7750d5-3059-4aa8-bb94-bdd2a3bbf23f None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:24.456788 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7750d5-3059-4aa8-bb94-bdd2a3bbf23f None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:24.457086 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7750d5-3059-4aa8-bb94-bdd2a3bbf23f None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:24.458920 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 334/656] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:03:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:03:24.459245 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7750d5-3059-4aa8-bb94-bdd2a3bbf23f None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:24.513462 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e34a01-8603-4fa8-bb3b-e031c2d753fe None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:24.537216 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:24.553494 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af32e8ad-5f88-4d66-b25a-5d705ef8fbc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:24.554136 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af32e8ad-5f88-4d66-b25a-5d705ef8fbc4 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:24.554416 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af32e8ad-5f88-4d66-b25a-5d705ef8fbc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:24.554687 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af32e8ad-5f88-4d66-b25a-5d705ef8fbc4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:24.562428 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:24.565640 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af32e8ad-5f88-4d66-b25a-5d705ef8fbc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:24.567093 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af32e8ad-5f88-4d66-b25a-5d705ef8fbc4 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:24.570463 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7750d5-3059-4aa8-bb94-bdd2a3bbf23f None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:24.583730 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-af32e8ad-5f88-4d66-b25a-5d705ef8fbc4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:24.583730 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:24.583730 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:24.583730 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:24.583730 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:24.583730 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:24.583730 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:24.583730 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:24.583730 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:24.583730 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:24.583730 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:24.583730 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:24.583730 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:03:24.583730 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:03:24.583730 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:03:24.584693 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:03:24.584693 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:24.584693 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:24.584693 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:24.584693 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:24.584693 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:24.584693 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:24.584693 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:24.584693 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:24.584693 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:24.584693 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:24.584693 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:24.584693 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:24.584693 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:24.584693 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:24.584693 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:24.584693 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:24.586302 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:03:24.586302 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:03:24.586302 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:24.586302 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:24.586302 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 323/657] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:03:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:24.586302 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af32e8ad-5f88-4d66-b25a-5d705ef8fbc4 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:24.614376 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2157b8-dd12-4b86-9b2c-0108149f7f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:24.615008 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2157b8-dd12-4b86-9b2c-0108149f7f7a None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:24.615295 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2157b8-dd12-4b86-9b2c-0108149f7f7a None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:24.615609 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2157b8-dd12-4b86-9b2c-0108149f7f7a None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:24.616300 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2157b8-dd12-4b86-9b2c-0108149f7f7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:24.616605 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2157b8-dd12-4b86-9b2c-0108149f7f7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:24.617971 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2157b8-dd12-4b86-9b2c-0108149f7f7a None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:24.629952 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 335/658] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:03:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:24.631139 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdd9d83-1921-405d-9f86-73143c284cda None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:24.631528 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdd9d83-1921-405d-9f86-73143c284cda None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:24.631748 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdd9d83-1921-405d-9f86-73143c284cda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:24.650429 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-dcdd9d83-1921-405d-9f86-73143c284cda None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:03:24.651668 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-dcdd9d83-1921-405d-9f86-73143c284cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:03:24.696157 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af32e8ad-5f88-4d66-b25a-5d705ef8fbc4 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:24.726898 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 336/659] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:03:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:03:24.727768 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdd9d83-1921-405d-9f86-73143c284cda None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:24.737301 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e902a8-8d81-423b-9645-6694707654b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:24.738153 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e902a8-8d81-423b-9645-6694707654b8 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:24.738632 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e902a8-8d81-423b-9645-6694707654b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:24.739086 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e902a8-8d81-423b-9645-6694707654b8 None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:24.740115 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e902a8-8d81-423b-9645-6694707654b8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:24.740670 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e902a8-8d81-423b-9645-6694707654b8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:24.742402 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e902a8-8d81-423b-9645-6694707654b8 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:24.762012 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 324/660] 10.222.0.25 () {56 vars in 1137 bytes} [Thu Aug 4 21:03:24 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:24.762963 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e902a8-8d81-423b-9645-6694707654b8 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:24.794145 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:24.828873 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:24.847441 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdd9d83-1921-405d-9f86-73143c284cda None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:24.868638 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e902a8-8d81-423b-9645-6694707654b8 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:24.884092 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e52d660-e2c0-4976-b18f-7722c6e049b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:24.884718 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e52d660-e2c0-4976-b18f-7722c6e049b0 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:24.884996 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e52d660-e2c0-4976-b18f-7722c6e049b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:24.885264 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e52d660-e2c0-4976-b18f-7722c6e049b0 None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:24.886069 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e52d660-e2c0-4976-b18f-7722c6e049b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:24.886538 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e52d660-e2c0-4976-b18f-7722c6e049b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:24.887921 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e52d660-e2c0-4976-b18f-7722c6e049b0 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:24.904557 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 337/661] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:03:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:24.904557 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e52d660-e2c0-4976-b18f-7722c6e049b0 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:24.907304 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e12bc4-5f60-4367-b4f6-3c13cf27e442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:24.907977 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e12bc4-5f60-4367-b4f6-3c13cf27e442 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:24.909052 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e12bc4-5f60-4367-b4f6-3c13cf27e442 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:24.909330 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e12bc4-5f60-4367-b4f6-3c13cf27e442 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:24.916441 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e12bc4-5f60-4367-b4f6-3c13cf27e442 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:24.917888 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e12bc4-5f60-4367-b4f6-3c13cf27e442 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:24.933342 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-d6e12bc4-5f60-4367-b4f6-3c13cf27e442 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 04 21:03:24.933342 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:24.933342 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:24.933342 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:24.933342 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:24.933342 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:24.933342 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:24.933342 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:24.933342 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:24.933342 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:24.933342 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:24.933342 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:24.933342 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 21:03:24.933342 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 21:03:24.933342 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 21:03:24.933342 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 21:03:24.934314 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:24.934314 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:24.934314 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 21:03:24.934314 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:03:24.934314 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 21:03:24.934314 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:03:24.934314 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:24.934314 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:24.934314 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:24.934314 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:24.934314 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:24.934314 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:24.934314 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:24.934314 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:24.934314 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:24.934314 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:24.934314 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:24.934314 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:24.935269 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:24.935269 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:24.935269 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 21:03:24.935269 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 21:03:24.935269 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 21:03:24.935269 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 21:03:24.935269 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 21:03:24.935269 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 21:03:24.935269 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 04 21:03:24.935269 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:24.935269 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 325/662] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Aug 4 21:03:24 2022] GET /identity/v3/users/neutron => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:24.935269 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e12bc4-5f60-4367-b4f6-3c13cf27e442 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:25.044775 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:25.051062 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e52d660-e2c0-4976-b18f-7722c6e049b0 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:25.055808 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e12bc4-5f60-4367-b4f6-3c13cf27e442 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:25.088400 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f56312f-aa6d-4e5a-a668-d8c48f06e4d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:25.089028 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f56312f-aa6d-4e5a-a668-d8c48f06e4d6 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:25.089320 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f56312f-aa6d-4e5a-a668-d8c48f06e4d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:25.089597 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f56312f-aa6d-4e5a-a668-d8c48f06e4d6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:25.098708 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862ae8ea-7416-4ab6-b248-1cd977fc0e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:25.099358 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862ae8ea-7416-4ab6-b248-1cd977fc0e4f None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:25.099663 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862ae8ea-7416-4ab6-b248-1cd977fc0e4f None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:25.099938 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862ae8ea-7416-4ab6-b248-1cd977fc0e4f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:25.101032 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f56312f-aa6d-4e5a-a668-d8c48f06e4d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:03:25.106835 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-4f56312f-aa6d-4e5a-a668-d8c48f06e4d6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:03:25.106835 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:25.106835 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:25.106835 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:25.106835 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:25.106835 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:25.106835 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:25.106835 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:25.106835 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:25.106835 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:25.106835 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:25.106835 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:25.106835 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:03:25.106835 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:03:25.106835 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:03:25.106835 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:03:25.108139 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:03:25.108139 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:03:25.108139 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:03:25.108139 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:03:25.108139 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:25.108139 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:25.108139 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:25.108139 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:25.108139 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:25.108139 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:25.108139 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:25.108139 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:25.108139 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:25.108139 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:25.108139 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:25.108139 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:25.108966 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:25.108966 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:25.108966 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:25.108966 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:25.108966 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:03:25.108966 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:03:25.108966 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:03:25.108966 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:03:25.108966 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:03:25.108966 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:03:25.108966 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:03:25.108966 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:25.108966 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 338/663] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Aug 4 21:03:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:25.108966 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f56312f-aa6d-4e5a-a668-d8c48f06e4d6 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:25.113883 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:25.192693 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862ae8ea-7416-4ab6-b248-1cd977fc0e4f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:25.195233 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862ae8ea-7416-4ab6-b248-1cd977fc0e4f None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:25.261872 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f56312f-aa6d-4e5a-a668-d8c48f06e4d6 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:25.302483 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bcf7c1-1a7c-4999-b737-5be25f2eecd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:25.303547 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bcf7c1-1a7c-4999-b737-5be25f2eecd3 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:25.304092 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bcf7c1-1a7c-4999-b737-5be25f2eecd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:25.304541 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bcf7c1-1a7c-4999-b737-5be25f2eecd3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:25.312437 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bcf7c1-1a7c-4999-b737-5be25f2eecd3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:25.315087 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:25.315435 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bcf7c1-1a7c-4999-b737-5be25f2eecd3 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:25.325733 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 326/664] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Aug 4 21:03:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 21:03:25.326646 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862ae8ea-7416-4ab6-b248-1cd977fc0e4f None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:25.328391 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-27bcf7c1-1a7c-4999-b737-5be25f2eecd3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:03:25.328391 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:25.328391 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:25.328391 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:25.328391 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:25.328391 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:25.328391 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:25.328391 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:25.328391 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:25.328391 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:25.328391 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:25.328391 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:25.328391 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 21:03:25.328391 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 21:03:25.328391 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 21:03:25.329264 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 21:03:25.329264 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:25.329264 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:25.329264 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:25.329264 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:25.329264 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:25.329264 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:25.329264 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:25.329264 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:25.329264 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:25.329264 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:25.329264 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:25.329264 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:25.329264 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:25.329264 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:25.329264 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:25.329264 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:25.330193 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 21:03:25.330193 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 21:03:25.330193 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 21:03:25.330193 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 21:03:25.330193 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 21:03:25.330193 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 21:03:25.330193 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:03:25.330193 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:25.332537 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 339/665] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Aug 4 21:03:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:25.336575 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bcf7c1-1a7c-4999-b737-5be25f2eecd3 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:25.413507 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:25.440123 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862ae8ea-7416-4ab6-b248-1cd977fc0e4f None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:25.463038 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bcf7c1-1a7c-4999-b737-5be25f2eecd3 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:25.485429 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073741c9-5773-4d3e-85b9-39593db78473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:25.486755 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073741c9-5773-4d3e-85b9-39593db78473 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:25.487040 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073741c9-5773-4d3e-85b9-39593db78473 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:25.487338 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073741c9-5773-4d3e-85b9-39593db78473 None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:25.488172 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073741c9-5773-4d3e-85b9-39593db78473 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:25.488477 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073741c9-5773-4d3e-85b9-39593db78473 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:25.489957 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073741c9-5773-4d3e-85b9-39593db78473 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:25.501138 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a25c3b-a11b-471b-9230-04b8d0d315fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:25.501958 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a25c3b-a11b-471b-9230-04b8d0d315fc None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:25.502890 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a25c3b-a11b-471b-9230-04b8d0d315fc None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:25.503255 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a25c3b-a11b-471b-9230-04b8d0d315fc None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:25.504087 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a25c3b-a11b-471b-9230-04b8d0d315fc None admin] RBAC: Adding query filter params (name=service) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:25.504932 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a25c3b-a11b-471b-9230-04b8d0d315fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:25.506069 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a25c3b-a11b-471b-9230-04b8d0d315fc None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:25.512377 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 327/666] 10.222.0.25 () {56 vars in 1139 bytes} [Thu Aug 4 21:03:25 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:25.512377 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073741c9-5773-4d3e-85b9-39593db78473 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:25.524058 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 340/667] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Aug 4 21:03:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:25.524058 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a25c3b-a11b-471b-9230-04b8d0d315fc None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:25.566467 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:25.620428 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073741c9-5773-4d3e-85b9-39593db78473 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:25.627557 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a25c3b-a11b-471b-9230-04b8d0d315fc None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:25.652870 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:25.660743 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee58ef4-884f-402f-a2a3-c8e644afd1be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:25.661682 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee58ef4-884f-402f-a2a3-c8e644afd1be None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:25.662058 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee58ef4-884f-402f-a2a3-c8e644afd1be None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:25.662513 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee58ef4-884f-402f-a2a3-c8e644afd1be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:25.663008 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ba0f3c-d785-427e-8030-f0983cc932f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:25.663702 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba0f3c-d785-427e-8030-f0983cc932f9 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:25.663934 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba0f3c-d785-427e-8030-f0983cc932f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:25.664208 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba0f3c-d785-427e-8030-f0983cc932f9 None admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:25.664981 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ba0f3c-d785-427e-8030-f0983cc932f9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:25.665283 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ba0f3c-d785-427e-8030-f0983cc932f9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:25.666710 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ba0f3c-d785-427e-8030-f0983cc932f9 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:25.674565 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee58ef4-884f-402f-a2a3-c8e644afd1be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:25.676198 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee58ef4-884f-402f-a2a3-c8e644afd1be None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:25.683615 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 341/668] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Aug 4 21:03:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:25.684458 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ba0f3c-d785-427e-8030-f0983cc932f9 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:25.691555 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-5ee58ef4-884f-402f-a2a3-c8e644afd1be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:25.691555 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:25.691555 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:25.691555 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:25.691555 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:25.691555 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:25.691555 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:25.691555 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:25.691555 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:25.691555 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:25.691555 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:25.691555 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:25.691555 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:03:25.691555 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:03:25.691555 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:03:25.692595 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:03:25.692595 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:25.692595 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:25.692595 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:25.692595 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:25.692595 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:25.692595 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:25.692595 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:25.692595 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:25.692595 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:25.692595 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:25.692595 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:25.692595 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:25.692595 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:25.692595 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:25.692595 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:25.692595 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:25.693518 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:03:25.693518 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:03:25.693518 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:25.693518 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:25.693518 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 328/669] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:03:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:25.693792 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee58ef4-884f-402f-a2a3-c8e644afd1be None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:25.789964 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:25.793466 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee58ef4-884f-402f-a2a3-c8e644afd1be None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:25.796241 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ba0f3c-d785-427e-8030-f0983cc932f9 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:25.828570 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d6c3d1-4761-41ed-bcea-16f5a6f5039b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:25.829204 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d6c3d1-4761-41ed-bcea-16f5a6f5039b None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:25.829486 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d6c3d1-4761-41ed-bcea-16f5a6f5039b None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:25.829756 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d6c3d1-4761-41ed-bcea-16f5a6f5039b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:25.833236 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26949ef-6331-42b6-a886-5aced5d59453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:25.833930 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26949ef-6331-42b6-a886-5aced5d59453 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:25.834287 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26949ef-6331-42b6-a886-5aced5d59453 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:25.834633 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26949ef-6331-42b6-a886-5aced5d59453 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:25.839676 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d6c3d1-4761-41ed-bcea-16f5a6f5039b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:25.841194 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d6c3d1-4761-41ed-bcea-16f5a6f5039b None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:25.856124 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-93d6c3d1-4761-41ed-bcea-16f5a6f5039b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:25.856124 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:25.856124 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:25.856124 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:25.856124 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:25.856124 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:25.856124 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:25.856124 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:25.856124 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:25.856124 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:25.856124 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:25.856124 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:25.856124 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:03:25.856124 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:03:25.856124 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:03:25.857199 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:03:25.857199 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:25.857199 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:25.857199 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:25.857199 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:25.857199 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:25.857199 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:25.857199 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:25.857199 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:25.857199 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:25.857199 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:25.857199 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:25.857199 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:25.857199 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:25.857199 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:25.857199 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:25.857199 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:25.858112 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:03:25.858112 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:03:25.858112 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:25.858112 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:25.858112 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 329/670] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:03:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:25.858511 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d6c3d1-4761-41ed-bcea-16f5a6f5039b None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:25.898680 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:25.913318 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26949ef-6331-42b6-a886-5aced5d59453 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:25.915180 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26949ef-6331-42b6-a886-5aced5d59453 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:25.962803 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d6c3d1-4761-41ed-bcea-16f5a6f5039b None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:25.997986 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6536a3a4-6cbc-4f5f-9dc5-ef78dbae25da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:25.998587 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6536a3a4-6cbc-4f5f-9dc5-ef78dbae25da None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:25.998865 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6536a3a4-6cbc-4f5f-9dc5-ef78dbae25da None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:25.999136 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6536a3a4-6cbc-4f5f-9dc5-ef78dbae25da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:25.999944 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6536a3a4-6cbc-4f5f-9dc5-ef78dbae25da None admin] RBAC: Adding query filter params (role.id=4eb30338d61e47808b51e050836643ea, scope.project.id=17dc3d39e2c048ca9070588f5717a578, user.id=e31f2728056b460fa474478eb85ca1d8) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:26.000247 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6536a3a4-6cbc-4f5f-9dc5-ef78dbae25da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:26.001767 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6536a3a4-6cbc-4f5f-9dc5-ef78dbae25da None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:26.019378 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 330/671] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Aug 4 21:03:25 2022] GET /identity/v3/role_assignments?user.id=e31f2728056b460fa474478eb85ca1d8&scope.project.id=17dc3d39e2c048ca9070588f5717a578&role.id=4eb30338d61e47808b51e050836643ea => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:26.019378 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6536a3a4-6cbc-4f5f-9dc5-ef78dbae25da None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:26.030838 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:26.043683 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 342/672] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Aug 4 21:03:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 21:03:26.045618 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26949ef-6331-42b6-a886-5aced5d59453 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:26.136645 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6536a3a4-6cbc-4f5f-9dc5-ef78dbae25da None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:26.174666 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:26.179622 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912dbf6b-6bc8-45bc-91f4-32996ee08fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:26.180475 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912dbf6b-6bc8-45bc-91f4-32996ee08fec None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:26.180918 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912dbf6b-6bc8-45bc-91f4-32996ee08fec None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:26.181356 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912dbf6b-6bc8-45bc-91f4-32996ee08fec None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:26.182209 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912dbf6b-6bc8-45bc-91f4-32996ee08fec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:26.183402 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912dbf6b-6bc8-45bc-91f4-32996ee08fec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:26.185008 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912dbf6b-6bc8-45bc-91f4-32996ee08fec None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:26.192115 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26949ef-6331-42b6-a886-5aced5d59453 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:26.201416 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 331/673] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:03:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:26.202281 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912dbf6b-6bc8-45bc-91f4-32996ee08fec None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:26.237877 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a364634-6cbd-4bec-9057-e8305b4c28bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:26.238514 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a364634-6cbd-4bec-9057-e8305b4c28bb None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:26.238801 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a364634-6cbd-4bec-9057-e8305b4c28bb None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:26.239072 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a364634-6cbd-4bec-9057-e8305b4c28bb None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:26.240670 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a364634-6cbd-4bec-9057-e8305b4c28bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:26.240982 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a364634-6cbd-4bec-9057-e8305b4c28bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:26.242377 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a364634-6cbd-4bec-9057-e8305b4c28bb None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:26.255098 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 343/674] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:03:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:26.260157 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a364634-6cbd-4bec-9057-e8305b4c28bb None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:26.303818 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:26.331665 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912dbf6b-6bc8-45bc-91f4-32996ee08fec None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:26.370222 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f225505d-b479-46fc-b8be-3319ede1c958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:26.370867 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f225505d-b479-46fc-b8be-3319ede1c958 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:26.371149 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f225505d-b479-46fc-b8be-3319ede1c958 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:26.371453 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f225505d-b479-46fc-b8be-3319ede1c958 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:26.382591 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f225505d-b479-46fc-b8be-3319ede1c958 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:26.384093 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f225505d-b479-46fc-b8be-3319ede1c958 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:26.398979 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a364634-6cbd-4bec-9057-e8305b4c28bb None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:26.399353 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-f225505d-b479-46fc-b8be-3319ede1c958 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:26.399353 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:26.399353 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:26.399353 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:26.399353 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:26.399353 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:26.399353 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:26.399353 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:26.399353 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:26.399353 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:26.399353 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:26.399353 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:26.399353 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:03:26.399353 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:03:26.399353 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:03:26.400375 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:03:26.400375 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:26.400375 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:26.400375 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:26.400375 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:26.400375 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:26.400375 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:26.400375 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:26.400375 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:26.400375 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:26.400375 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:26.400375 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:26.400375 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:26.400375 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:26.400375 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:26.400375 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:26.400375 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:26.401582 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:03:26.401582 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:03:26.401582 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:26.401582 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:26.401582 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 332/675] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:03:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:26.401582 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f225505d-b479-46fc-b8be-3319ede1c958 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:26.436792 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6c1064-786c-48cd-8cd8-048115d47737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:26.437360 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6c1064-786c-48cd-8cd8-048115d47737 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:26.437677 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6c1064-786c-48cd-8cd8-048115d47737 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:26.437906 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6c1064-786c-48cd-8cd8-048115d47737 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:26.444990 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6c1064-786c-48cd-8cd8-048115d47737 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:26.446424 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6c1064-786c-48cd-8cd8-048115d47737 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:26.458125 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-ac6c1064-786c-48cd-8cd8-048115d47737 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:03:26.458125 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:26.458125 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:26.458125 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:26.458125 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:26.458125 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:26.458125 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:26.458125 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:26.458125 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:26.458125 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:26.458125 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:26.458125 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:26.458125 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 21:03:26.458125 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 21:03:26.458125 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 21:03:26.459202 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 21:03:26.459202 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:26.459202 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:26.459202 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:26.459202 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:26.459202 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:26.459202 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:26.459202 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:26.459202 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:26.459202 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:26.459202 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:26.459202 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:26.459202 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:26.459202 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:26.459202 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:26.459202 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:26.459202 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:26.460151 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 21:03:26.460151 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 21:03:26.460151 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 21:03:26.460151 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 21:03:26.460151 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 21:03:26.460151 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 21:03:26.460151 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:03:26.460151 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:26.460151 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 344/676] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Aug 4 21:03:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:26.460850 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ebd17f-48ce-4fa0-87c4-6d04058ab90a None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:26.461140 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ebd17f-48ce-4fa0-87c4-6d04058ab90a None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:26.461410 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ebd17f-48ce-4fa0-87c4-6d04058ab90a None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:26.467000 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 345/677] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:03:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:03:26.467784 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ebd17f-48ce-4fa0-87c4-6d04058ab90a None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:26.478532 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:26.514999 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f225505d-b479-46fc-b8be-3319ede1c958 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:26.544941 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:26.554492 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c453c46-c111-4c0b-a71d-1d34a935bcda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:26.555132 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c453c46-c111-4c0b-a71d-1d34a935bcda None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:26.555441 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c453c46-c111-4c0b-a71d-1d34a935bcda None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:26.555715 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c453c46-c111-4c0b-a71d-1d34a935bcda None admin] PATH_INFO: `/v3/users/swift` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:26.563965 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c453c46-c111-4c0b-a71d-1d34a935bcda None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:26.565591 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c453c46-c111-4c0b-a71d-1d34a935bcda None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:26.572904 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ebd17f-48ce-4fa0-87c4-6d04058ab90a None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:26.577598 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-3c453c46-c111-4c0b-a71d-1d34a935bcda None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 04 21:03:26.577598 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:26.577598 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:26.577598 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:26.577598 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:26.577598 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:26.577598 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:26.577598 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:26.577598 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:26.577598 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:26.577598 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:26.577598 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:26.577598 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 21:03:26.577598 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 21:03:26.577598 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 21:03:26.577598 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 21:03:26.578550 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:26.578550 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:26.578550 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 21:03:26.578550 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:03:26.578550 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 21:03:26.578550 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:03:26.578550 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:26.578550 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:26.578550 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:26.578550 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:26.578550 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:26.578550 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:26.578550 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:26.578550 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:26.578550 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:26.578550 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:26.578550 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:26.578550 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:26.579551 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:26.579551 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:26.579551 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 21:03:26.579551 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 21:03:26.579551 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 21:03:26.579551 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 21:03:26.579551 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 21:03:26.579551 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 21:03:26.579551 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 04 21:03:26.579551 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:26.583300 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 333/678] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Aug 4 21:03:26 2022] GET /identity/v3/users/swift => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:26.584140 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c453c46-c111-4c0b-a71d-1d34a935bcda None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:26.609986 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cc6527-7690-4cd2-adf7-2505bf89beee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:26.610633 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cc6527-7690-4cd2-adf7-2505bf89beee None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:26.610881 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cc6527-7690-4cd2-adf7-2505bf89beee None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:26.611171 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cc6527-7690-4cd2-adf7-2505bf89beee None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:26.612510 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cc6527-7690-4cd2-adf7-2505bf89beee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:26.612812 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cc6527-7690-4cd2-adf7-2505bf89beee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:26.614179 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cc6527-7690-4cd2-adf7-2505bf89beee None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:26.626488 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 346/679] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:03:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:26.627756 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5ec6f7-eb2d-45c3-bc92-8b0942c23105 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:26.628046 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5ec6f7-eb2d-45c3-bc92-8b0942c23105 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:26.628319 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5ec6f7-eb2d-45c3-bc92-8b0942c23105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:26.647148 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-2a5ec6f7-eb2d-45c3-bc92-8b0942c23105 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:03:26.648368 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-2a5ec6f7-eb2d-45c3-bc92-8b0942c23105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:03:26.698120 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c453c46-c111-4c0b-a71d-1d34a935bcda None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:26.710215 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:26.718435 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 347/680] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:03:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:03:26.719269 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5ec6f7-eb2d-45c3-bc92-8b0942c23105 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:26.745009 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21241210-c45c-4889-a5f6-24ef44317982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:26.745700 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21241210-c45c-4889-a5f6-24ef44317982 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:26.745986 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21241210-c45c-4889-a5f6-24ef44317982 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:26.746259 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21241210-c45c-4889-a5f6-24ef44317982 None admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:26.747094 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21241210-c45c-4889-a5f6-24ef44317982 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:26.747435 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21241210-c45c-4889-a5f6-24ef44317982 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:26.749041 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21241210-c45c-4889-a5f6-24ef44317982 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:26.769144 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 334/681] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Aug 4 21:03:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:26.771130 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21241210-c45c-4889-a5f6-24ef44317982 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:26.822763 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:26.842490 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5ec6f7-eb2d-45c3-bc92-8b0942c23105 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:26.877590 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21241210-c45c-4889-a5f6-24ef44317982 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:26.877900 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b65ff2-364e-4641-a3c5-4577d3b41f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:26.878463 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b65ff2-364e-4641-a3c5-4577d3b41f0b None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:26.878739 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b65ff2-364e-4641-a3c5-4577d3b41f0b None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:26.879015 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b65ff2-364e-4641-a3c5-4577d3b41f0b None admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:26.879830 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b65ff2-364e-4641-a3c5-4577d3b41f0b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:26.880135 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b65ff2-364e-4641-a3c5-4577d3b41f0b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:26.881630 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b65ff2-364e-4641-a3c5-4577d3b41f0b None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:26.903146 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 348/682] 10.222.0.25 () {56 vars in 1135 bytes} [Thu Aug 4 21:03:26 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:26.903146 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b65ff2-364e-4641-a3c5-4577d3b41f0b None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:26.917523 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab94180-d766-4a3e-bb9e-d5dd76f2b21a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:26.918168 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab94180-d766-4a3e-bb9e-d5dd76f2b21a None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:26.918456 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab94180-d766-4a3e-bb9e-d5dd76f2b21a None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:26.918730 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab94180-d766-4a3e-bb9e-d5dd76f2b21a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:26.925992 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab94180-d766-4a3e-bb9e-d5dd76f2b21a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:26.927478 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab94180-d766-4a3e-bb9e-d5dd76f2b21a None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:26.939044 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-cab94180-d766-4a3e-bb9e-d5dd76f2b21a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:03:26.939044 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:26.939044 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:26.939044 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:26.939044 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:26.939044 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:26.939044 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:26.939044 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:26.939044 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:26.939044 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:26.939044 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:26.939044 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:26.939044 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 21:03:26.939044 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 21:03:26.939044 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 21:03:26.940685 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 21:03:26.940685 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:26.940685 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:26.940685 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:26.940685 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:26.940685 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:26.940685 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:26.940685 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:26.940685 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:26.940685 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:26.940685 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:26.940685 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:26.940685 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:26.940685 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:26.940685 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:26.940685 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:26.940685 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:26.941646 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 21:03:26.941646 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 21:03:26.941646 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 21:03:26.941646 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 21:03:26.941646 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 21:03:26.941646 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 21:03:26.941646 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:03:26.941646 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:26.941646 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 335/683] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Aug 4 21:03:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:26.942185 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab94180-d766-4a3e-bb9e-d5dd76f2b21a None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:26.963484 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:27.012177 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b65ff2-364e-4641-a3c5-4577d3b41f0b None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:27.045061 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab94180-d766-4a3e-bb9e-d5dd76f2b21a None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:27.046497 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0ed68e-9f74-41f2-8c28-8d81e3f03dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:27.047136 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0ed68e-9f74-41f2-8c28-8d81e3f03dc5 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:27.047474 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0ed68e-9f74-41f2-8c28-8d81e3f03dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:27.047745 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0ed68e-9f74-41f2-8c28-8d81e3f03dc5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:27.055445 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:27.058825 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0ed68e-9f74-41f2-8c28-8d81e3f03dc5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:03:27.064209 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-9e0ed68e-9f74-41f2-8c28-8d81e3f03dc5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:03:27.064209 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:27.064209 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:27.064209 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:27.064209 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:27.064209 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:27.064209 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:27.064209 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:27.064209 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:27.064209 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:27.064209 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:27.064209 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:27.064209 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:03:27.064209 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:03:27.064209 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:03:27.064209 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:03:27.068735 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:03:27.068735 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:03:27.068735 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:03:27.068735 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:03:27.068735 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:27.068735 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:27.068735 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:27.068735 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:27.068735 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:27.068735 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:27.068735 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:27.068735 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:27.068735 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:27.068735 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:27.068735 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:27.068735 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:27.069580 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:27.069580 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:27.069580 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:27.069580 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:27.069580 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:03:27.069580 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:03:27.069580 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:03:27.069580 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:03:27.069580 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:03:27.069580 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:03:27.069580 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:03:27.069580 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:27.069580 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 349/684] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Aug 4 21:03:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:27.069580 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0ed68e-9f74-41f2-8c28-8d81e3f03dc5 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:27.085159 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6187475-52e8-43ef-b80c-2b98c19873a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:27.086057 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6187475-52e8-43ef-b80c-2b98c19873a2 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:27.086552 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6187475-52e8-43ef-b80c-2b98c19873a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:27.087073 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6187475-52e8-43ef-b80c-2b98c19873a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:27.088080 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6187475-52e8-43ef-b80c-2b98c19873a2 None admin] RBAC: Adding query filter params (role.id=4eb30338d61e47808b51e050836643ea, scope.project.id=17dc3d39e2c048ca9070588f5717a578, user.id=f6885c7d929f4f8e96d3890724c3b53f) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:27.088390 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6187475-52e8-43ef-b80c-2b98c19873a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:27.089831 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6187475-52e8-43ef-b80c-2b98c19873a2 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:27.104282 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 336/685] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Aug 4 21:03:26 2022] GET /identity/v3/role_assignments?user.id=f6885c7d929f4f8e96d3890724c3b53f&scope.project.id=17dc3d39e2c048ca9070588f5717a578&role.id=4eb30338d61e47808b51e050836643ea => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:27.104282 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6187475-52e8-43ef-b80c-2b98c19873a2 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:27.203272 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0ed68e-9f74-41f2-8c28-8d81e3f03dc5 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:27.228126 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:27.246653 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6187475-52e8-43ef-b80c-2b98c19873a2 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:27.253566 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79b991e-271d-4a00-b483-f5e72572396d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:27.253566 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b991e-271d-4a00-b483-f5e72572396d None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:27.253566 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b991e-271d-4a00-b483-f5e72572396d None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:27.253566 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b991e-271d-4a00-b483-f5e72572396d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:27.297371 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f62f30-1b1d-4a87-9ad8-f0dc45a6a89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:27.299170 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f62f30-1b1d-4a87-9ad8-f0dc45a6a89d None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:27.299489 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f62f30-1b1d-4a87-9ad8-f0dc45a6a89d None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:27.299820 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f62f30-1b1d-4a87-9ad8-f0dc45a6a89d None admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:27.302036 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f62f30-1b1d-4a87-9ad8-f0dc45a6a89d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:27.302036 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f62f30-1b1d-4a87-9ad8-f0dc45a6a89d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:27.302770 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f62f30-1b1d-4a87-9ad8-f0dc45a6a89d None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:27.330876 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 337/686] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Aug 4 21:03:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:27.331812 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f62f30-1b1d-4a87-9ad8-f0dc45a6a89d None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:27.339723 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79b991e-271d-4a00-b483-f5e72572396d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:27.341191 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79b991e-271d-4a00-b483-f5e72572396d None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:27.380127 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:27.458594 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 350/687] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Aug 4 21:03:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 21:03:27.459289 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79b991e-271d-4a00-b483-f5e72572396d None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:27.465197 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f62f30-1b1d-4a87-9ad8-f0dc45a6a89d None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:27.474407 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:27.503479 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf62e00-4026-4d64-bcdd-094d3fceaf53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:27.504422 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf62e00-4026-4d64-bcdd-094d3fceaf53 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:27.504935 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf62e00-4026-4d64-bcdd-094d3fceaf53 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:27.505441 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf62e00-4026-4d64-bcdd-094d3fceaf53 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:27.507292 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf62e00-4026-4d64-bcdd-094d3fceaf53 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:27.507849 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf62e00-4026-4d64-bcdd-094d3fceaf53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:27.509551 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf62e00-4026-4d64-bcdd-094d3fceaf53 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:27.519894 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 338/688] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Aug 4 21:03:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:27.521060 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf62e00-4026-4d64-bcdd-094d3fceaf53 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:27.561967 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79b991e-271d-4a00-b483-f5e72572396d None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:27.596631 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3653b5c4-e076-431e-9bc9-6b63197755cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:27.597067 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3653b5c4-e076-431e-9bc9-6b63197755cd None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:27.597352 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3653b5c4-e076-431e-9bc9-6b63197755cd None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:27.597620 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3653b5c4-e076-431e-9bc9-6b63197755cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:27.598466 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3653b5c4-e076-431e-9bc9-6b63197755cd None admin] RBAC: Adding query filter params (role.id=4eb30338d61e47808b51e050836643ea, scope.project.id=17dc3d39e2c048ca9070588f5717a578, user.id=08fcb91ee54344b5bb3870c585e25fdc) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:27.598773 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3653b5c4-e076-431e-9bc9-6b63197755cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:27.600228 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3653b5c4-e076-431e-9bc9-6b63197755cd None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:27.611957 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 351/689] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Aug 4 21:03:27 2022] GET /identity/v3/role_assignments?user.id=08fcb91ee54344b5bb3870c585e25fdc&scope.project.id=17dc3d39e2c048ca9070588f5717a578&role.id=4eb30338d61e47808b51e050836643ea => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:27.612693 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3653b5c4-e076-431e-9bc9-6b63197755cd None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:27.629402 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:27.638165 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf62e00-4026-4d64-bcdd-094d3fceaf53 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:27.679528 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591c2017-fa0c-426a-a307-cb8523b83982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:27.680582 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591c2017-fa0c-426a-a307-cb8523b83982 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:27.681067 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591c2017-fa0c-426a-a307-cb8523b83982 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:27.681549 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591c2017-fa0c-426a-a307-cb8523b83982 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:27.693846 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591c2017-fa0c-426a-a307-cb8523b83982 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:27.695834 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591c2017-fa0c-426a-a307-cb8523b83982 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:27.713296 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-591c2017-fa0c-426a-a307-cb8523b83982 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:27.713296 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:27.713296 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:27.713296 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:27.713296 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:27.713296 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:27.713296 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:27.713296 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:27.713296 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:27.713296 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:27.713296 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:27.713296 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:27.713296 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:03:27.713296 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:03:27.713296 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:03:27.714288 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:03:27.714288 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:27.714288 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:27.714288 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:27.714288 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:27.714288 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:27.714288 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:27.714288 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:27.714288 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:27.714288 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:27.714288 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:27.714288 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:27.714288 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:27.714288 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:27.714288 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:27.714288 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:27.714288 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:27.715163 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:03:27.715163 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:03:27.715163 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:27.715163 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:27.719190 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 339/690] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:03:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:27.724462 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591c2017-fa0c-426a-a307-cb8523b83982 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:27.800155 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:27.838407 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591c2017-fa0c-426a-a307-cb8523b83982 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:27.873207 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3653b5c4-e076-431e-9bc9-6b63197755cd None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:27.879051 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ab4cba-5b35-4424-9888-3a8063ba9752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:27.880058 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ab4cba-5b35-4424-9888-3a8063ba9752 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:27.880562 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ab4cba-5b35-4424-9888-3a8063ba9752 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:27.881046 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ab4cba-5b35-4424-9888-3a8063ba9752 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:27.881955 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ab4cba-5b35-4424-9888-3a8063ba9752 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:27.882478 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ab4cba-5b35-4424-9888-3a8063ba9752 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:27.884140 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ab4cba-5b35-4424-9888-3a8063ba9752 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:27.898140 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 340/691] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:03:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:03:27.903286 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ab4cba-5b35-4424-9888-3a8063ba9752 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:27.915649 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ed98f-00ac-4a56-9737-1760367e03d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:27.916289 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ed98f-00ac-4a56-9737-1760367e03d8 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:27.916567 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ed98f-00ac-4a56-9737-1760367e03d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:27.916840 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ed98f-00ac-4a56-9737-1760367e03d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:27.929578 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19ed98f-00ac-4a56-9737-1760367e03d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:27.931198 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19ed98f-00ac-4a56-9737-1760367e03d8 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:27.942653 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:27.952067 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-e19ed98f-00ac-4a56-9737-1760367e03d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:27.952067 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:27.952067 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:27.952067 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:27.952067 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:27.952067 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:27.952067 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:27.952067 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:27.952067 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:27.952067 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:27.952067 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:27.952067 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:27.952067 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:03:27.952067 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:03:27.952067 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:03:27.953228 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:03:27.953228 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:27.953228 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:27.953228 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:27.953228 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:27.953228 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:27.953228 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:27.953228 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:27.953228 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:27.953228 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:27.953228 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:27.953228 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:27.953228 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:27.953228 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:27.953228 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:27.953228 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:27.953228 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:27.954316 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:03:27.954316 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:03:27.954316 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:27.954316 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:27.954316 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 352/692] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:03:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:27.961956 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ed98f-00ac-4a56-9737-1760367e03d8 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:28.034640 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ab4cba-5b35-4424-9888-3a8063ba9752 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:28.078736 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64272fac-1ebf-4945-8d6f-6e6c3b332022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:28.079344 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64272fac-1ebf-4945-8d6f-6e6c3b332022 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:28.079627 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64272fac-1ebf-4945-8d6f-6e6c3b332022 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:28.079931 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64272fac-1ebf-4945-8d6f-6e6c3b332022 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:28.088981 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64272fac-1ebf-4945-8d6f-6e6c3b332022 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:28.090751 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64272fac-1ebf-4945-8d6f-6e6c3b332022 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:28.095056 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ed98f-00ac-4a56-9737-1760367e03d8 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:28.104535 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-64272fac-1ebf-4945-8d6f-6e6c3b332022 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:03:28.104535 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:28.104535 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:28.104535 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:28.104535 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:28.104535 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:28.104535 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:28.104535 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:28.104535 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:28.104535 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:28.104535 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:28.104535 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:28.104535 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 21:03:28.104535 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 21:03:28.104535 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 21:03:28.105639 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 21:03:28.105639 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:28.105639 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:28.105639 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:28.105639 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:28.105639 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:28.105639 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:28.105639 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:28.105639 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:28.105639 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:28.105639 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:28.105639 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:28.105639 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:28.105639 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:28.105639 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:28.105639 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:28.105639 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:28.106693 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 21:03:28.106693 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 21:03:28.106693 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 21:03:28.106693 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 21:03:28.106693 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 21:03:28.106693 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 21:03:28.106693 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:03:28.106693 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:28.106693 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 341/693] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Aug 4 21:03:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:03:28.113814 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64272fac-1ebf-4945-8d6f-6e6c3b332022 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:28.144137 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:28.160704 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a5bc03-6093-4239-b1d8-5d479594d1fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:28.161513 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a5bc03-6093-4239-b1d8-5d479594d1fd None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:28.161856 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a5bc03-6093-4239-b1d8-5d479594d1fd None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:28.162141 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a5bc03-6093-4239-b1d8-5d479594d1fd None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:28.162843 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a5bc03-6093-4239-b1d8-5d479594d1fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:28.163296 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a5bc03-6093-4239-b1d8-5d479594d1fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:28.166394 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a5bc03-6093-4239-b1d8-5d479594d1fd None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:28.187718 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:28.190451 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 353/694] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:03:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:28.196173 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a5bc03-6093-4239-b1d8-5d479594d1fd None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:28.228403 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64272fac-1ebf-4945-8d6f-6e6c3b332022 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:28.272384 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df6aaea-7a66-4a59-a9c2-8391d213a4f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:28.273013 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df6aaea-7a66-4a59-a9c2-8391d213a4f9 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:28.273297 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df6aaea-7a66-4a59-a9c2-8391d213a4f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:28.273859 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df6aaea-7a66-4a59-a9c2-8391d213a4f9 None admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:28.275519 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df6aaea-7a66-4a59-a9c2-8391d213a4f9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:28.275836 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df6aaea-7a66-4a59-a9c2-8391d213a4f9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:28.277295 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df6aaea-7a66-4a59-a9c2-8391d213a4f9 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:28.295386 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 342/695] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Aug 4 21:03:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:28.297989 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b3d25b-fe75-4a84-aa73-fa212ea1ea3a None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:28.298500 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b3d25b-fe75-4a84-aa73-fa212ea1ea3a None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:28.299008 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b3d25b-fe75-4a84-aa73-fa212ea1ea3a None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:28.301015 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 343/696] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:03:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:03:28.302452 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b3d25b-fe75-4a84-aa73-fa212ea1ea3a None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:28.328524 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a5bc03-6093-4239-b1d8-5d479594d1fd None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:28.375734 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bf6667-c4c7-4b05-b35f-3710ab461ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:28.377433 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bf6667-c4c7-4b05-b35f-3710ab461ff4 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:28.378230 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bf6667-c4c7-4b05-b35f-3710ab461ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:28.378987 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bf6667-c4c7-4b05-b35f-3710ab461ff4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:28.388380 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bf6667-c4c7-4b05-b35f-3710ab461ff4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:28.390794 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bf6667-c4c7-4b05-b35f-3710ab461ff4 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:28.406375 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-33bf6667-c4c7-4b05-b35f-3710ab461ff4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 21:03:28.406375 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:28.406375 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:28.406375 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:28.406375 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:28.406375 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:28.406375 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:28.406375 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:28.406375 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:28.406375 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:28.406375 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:28.406375 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:28.406375 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 21:03:28.406375 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 21:03:28.406375 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 21:03:28.406375 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 21:03:28.407613 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:28.407613 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:28.407613 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 21:03:28.407613 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:03:28.407613 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 21:03:28.407613 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:03:28.407613 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:28.407613 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:28.407613 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:28.407613 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:28.407613 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:28.407613 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:28.407613 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:28.407613 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:28.407613 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:28.407613 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:28.407613 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:28.407613 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:28.409086 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:28.409086 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:28.409086 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 21:03:28.409086 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 21:03:28.409086 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 21:03:28.409086 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 21:03:28.409086 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 21:03:28.409086 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 21:03:28.409086 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 21:03:28.409086 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:28.411803 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 354/697] 10.222.0.25 () {56 vars in 1093 bytes} [Thu Aug 4 21:03:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:03:28.413179 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc3870f-79db-429b-af0e-5605f2c2564f None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:28.413614 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc3870f-79db-429b-af0e-5605f2c2564f None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:28.414045 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc3870f-79db-429b-af0e-5605f2c2564f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:28.429891 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b3d25b-fe75-4a84-aa73-fa212ea1ea3a None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:28.438540 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-9bc3870f-79db-429b-af0e-5605f2c2564f None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:03:28.440663 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-9bc3870f-79db-429b-af0e-5605f2c2564f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:03:28.463842 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:28.478012 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd3f580-a88e-4fd8-bdab-e4f6cbe103c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:28.486853 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd3f580-a88e-4fd8-bdab-e4f6cbe103c8 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:28.486853 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd3f580-a88e-4fd8-bdab-e4f6cbe103c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:28.486853 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd3f580-a88e-4fd8-bdab-e4f6cbe103c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:28.486853 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd3f580-a88e-4fd8-bdab-e4f6cbe103c8 None admin] RBAC: Adding query filter params (role.id=4eb30338d61e47808b51e050836643ea, scope.project.id=17dc3d39e2c048ca9070588f5717a578, user.id=58935367c0ad445f8477cdac0df04d0a) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:28.486853 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd3f580-a88e-4fd8-bdab-e4f6cbe103c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:28.486853 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd3f580-a88e-4fd8-bdab-e4f6cbe103c8 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:28.490275 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 344/698] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Aug 4 21:03:28 2022] GET /identity/v3/role_assignments?user.id=58935367c0ad445f8477cdac0df04d0a&scope.project.id=17dc3d39e2c048ca9070588f5717a578&role.id=4eb30338d61e47808b51e050836643ea => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:28.491154 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd3f580-a88e-4fd8-bdab-e4f6cbe103c8 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:28.507957 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:28.535647 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 355/699] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:03:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:03:28.581037 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc3870f-79db-429b-af0e-5605f2c2564f None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:28.700508 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc3870f-79db-429b-af0e-5605f2c2564f None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:28.732563 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd3f580-a88e-4fd8-bdab-e4f6cbe103c8 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:28.751190 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5e5fa8-4433-49f1-8ce6-7a656a689472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:28.751416 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5e5fa8-4433-49f1-8ce6-7a656a689472 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:28.751624 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5e5fa8-4433-49f1-8ce6-7a656a689472 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:28.752349 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5e5fa8-4433-49f1-8ce6-7a656a689472 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:28.752993 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5e5fa8-4433-49f1-8ce6-7a656a689472 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:28.754695 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5e5fa8-4433-49f1-8ce6-7a656a689472 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:28.770842 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-3f5e5fa8-4433-49f1-8ce6-7a656a689472 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 21:03:28.770842 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:28.770842 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 21:03:28.770842 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 21:03:28.770842 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 21:03:28.770842 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 21:03:28.770842 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 21:03:28.770842 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:03:28.770842 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 21:03:28.770842 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:28.770842 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 21:03:28.770842 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:28.770842 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:28.770842 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:28.770842 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:28.770842 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:28.772053 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:28.772053 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:28.772053 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:28.772053 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:28.772053 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:28.772053 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:28.772053 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:28.772053 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 21:03:28.772053 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 21:03:28.772053 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 21:03:28.772053 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 21:03:28.772053 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:28.772053 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:28.772053 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:28.772053 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:28.772053 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:28.772959 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:28.772959 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:28.772959 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:28.772959 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:28.772959 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:28.772959 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:28.772959 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:28.772959 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:28.772959 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:28.772959 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:28.772959 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:28.772959 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 21:03:28.772959 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:03:28.772959 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 21:03:28.772959 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:28.772959 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 356/700] 10.222.0.25 () {56 vars in 1129 bytes} [Thu Aug 4 21:03:28 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:28.778758 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5e5fa8-4433-49f1-8ce6-7a656a689472 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:28.785563 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5731315-851e-44d7-8bea-51dbd58f8be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:28.786549 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5731315-851e-44d7-8bea-51dbd58f8be4 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:28.787036 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5731315-851e-44d7-8bea-51dbd58f8be4 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:28.787551 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5731315-851e-44d7-8bea-51dbd58f8be4 None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:28.788600 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5731315-851e-44d7-8bea-51dbd58f8be4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:28.789197 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5731315-851e-44d7-8bea-51dbd58f8be4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:28.790951 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5731315-851e-44d7-8bea-51dbd58f8be4 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:28.812050 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 345/701] 10.222.0.25 () {56 vars in 1133 bytes} [Thu Aug 4 21:03:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:28.822391 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5731315-851e-44d7-8bea-51dbd58f8be4 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:28.838720 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:28.899702 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:28.920916 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5e5fa8-4433-49f1-8ce6-7a656a689472 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:28.969828 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20979e5a-2208-414d-9570-05ab070bacca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:28.971447 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20979e5a-2208-414d-9570-05ab070bacca None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:28.972197 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20979e5a-2208-414d-9570-05ab070bacca None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:28.972685 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20979e5a-2208-414d-9570-05ab070bacca None admin] PATH_INFO: `/v3/services` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:28.973596 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20979e5a-2208-414d-9570-05ab070bacca None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:28.973812 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5731315-851e-44d7-8bea-51dbd58f8be4 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:28.974313 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20979e5a-2208-414d-9570-05ab070bacca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:28.977155 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20979e5a-2208-414d-9570-05ab070bacca None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:28.983927 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 357/702] 10.222.0.25 () {56 vars in 1124 bytes} [Thu Aug 4 21:03:28 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:28.992296 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20979e5a-2208-414d-9570-05ab070bacca None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:29.021891 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adaf047-a1a2-432d-a843-4c8c61fad76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:29.022580 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adaf047-a1a2-432d-a843-4c8c61fad76a None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:29.022864 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adaf047-a1a2-432d-a843-4c8c61fad76a None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:29.023239 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adaf047-a1a2-432d-a843-4c8c61fad76a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:29.118712 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adaf047-a1a2-432d-a843-4c8c61fad76a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:29.121110 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adaf047-a1a2-432d-a843-4c8c61fad76a None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:29.126183 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20979e5a-2208-414d-9570-05ab070bacca None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:29.135913 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:29.184657 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3d838f-21a6-4f91-bed2-df51759e16fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:29.185294 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3d838f-21a6-4f91-bed2-df51759e16fd None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:29.185574 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3d838f-21a6-4f91-bed2-df51759e16fd None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:29.185849 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3d838f-21a6-4f91-bed2-df51759e16fd None admin] PATH_INFO: `/v3/services` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:29.186667 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3d838f-21a6-4f91-bed2-df51759e16fd None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:29.186812 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3d838f-21a6-4f91-bed2-df51759e16fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:29.188288 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3d838f-21a6-4f91-bed2-df51759e16fd None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:29.189656 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:29.203202 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 358/703] 10.222.0.25 () {56 vars in 1124 bytes} [Thu Aug 4 21:03:28 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:29.260842 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 346/704] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Aug 4 21:03:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 21:03:29.266813 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3d838f-21a6-4f91-bed2-df51759e16fd None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:29.388533 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3d838f-21a6-4f91-bed2-df51759e16fd None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:29.423622 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670168e0-f3f9-4ce8-8136-8e75e4434ea0 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:29.424113 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670168e0-f3f9-4ce8-8136-8e75e4434ea0 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:29.424311 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670168e0-f3f9-4ce8-8136-8e75e4434ea0 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:29.427339 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 347/705] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:03:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:03:29.429597 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41daf539-7968-4cf6-802b-18e73108ff67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:29.430683 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41daf539-7968-4cf6-802b-18e73108ff67 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:29.431124 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41daf539-7968-4cf6-802b-18e73108ff67 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:29.431625 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41daf539-7968-4cf6-802b-18e73108ff67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:29.432030 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfcf015-a3ad-4c52-963c-d6838e70892e None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:29.432376 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfcf015-a3ad-4c52-963c-d6838e70892e None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:29.432587 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfcf015-a3ad-4c52-963c-d6838e70892e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:29.449186 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41daf539-7968-4cf6-802b-18e73108ff67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:29.450964 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41daf539-7968-4cf6-802b-18e73108ff67 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:29.453368 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-bcfcf015-a3ad-4c52-963c-d6838e70892e None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:03:29.454693 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-bcfcf015-a3ad-4c52-963c-d6838e70892e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:03:29.467622 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-41daf539-7968-4cf6-802b-18e73108ff67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:29.467622 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:29.467622 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:29.467622 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:29.467622 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:29.467622 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:29.467622 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:29.467622 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:29.467622 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:29.467622 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:29.467622 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:29.467622 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:29.467622 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:03:29.467622 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:03:29.467622 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:03:29.468633 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:03:29.468633 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:29.468633 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:29.468633 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:29.468633 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:29.468633 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:29.468633 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:29.468633 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:29.468633 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:29.468633 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:29.468633 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:29.468633 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:29.468633 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:29.468633 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:29.468633 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:29.468633 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:29.468633 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:29.469540 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:03:29.469540 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:03:29.469540 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:29.469540 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:29.471906 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 359/706] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:03:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:29.484824 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41daf539-7968-4cf6-802b-18e73108ff67 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:29.518988 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:29.542075 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 348/707] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:03:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 21:03:29.566069 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfcf015-a3ad-4c52-963c-d6838e70892e None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:29.622022 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41daf539-7968-4cf6-802b-18e73108ff67 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:29.660011 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9483ac3a-24f3-4f5b-980e-9f541db1183d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:29.660648 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9483ac3a-24f3-4f5b-980e-9f541db1183d None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:29.661300 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9483ac3a-24f3-4f5b-980e-9f541db1183d None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:29.661300 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9483ac3a-24f3-4f5b-980e-9f541db1183d None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:29.661974 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9483ac3a-24f3-4f5b-980e-9f541db1183d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:29.662280 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9483ac3a-24f3-4f5b-980e-9f541db1183d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:29.663679 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9483ac3a-24f3-4f5b-980e-9f541db1183d None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:29.676616 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 360/708] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:03:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:29.681693 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9483ac3a-24f3-4f5b-980e-9f541db1183d None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:29.726629 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:29.777335 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfcf015-a3ad-4c52-963c-d6838e70892e None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:29.840422 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:29.845923 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dc0bf9-a974-45e3-ae7a-f3054118a017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:29.846581 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dc0bf9-a974-45e3-ae7a-f3054118a017 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:29.852818 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dc0bf9-a974-45e3-ae7a-f3054118a017 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:29.852818 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dc0bf9-a974-45e3-ae7a-f3054118a017 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:29.864504 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dc0bf9-a974-45e3-ae7a-f3054118a017 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:29.866778 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dc0bf9-a974-45e3-ae7a-f3054118a017 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:29.874285 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9483ac3a-24f3-4f5b-980e-9f541db1183d None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:29.895809 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-32dc0bf9-a974-45e3-ae7a-f3054118a017 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:29.895809 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:29.895809 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:29.895809 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:29.895809 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:29.895809 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:29.895809 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:29.895809 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:29.895809 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:29.895809 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:29.895809 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:29.895809 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:29.895809 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:03:29.895809 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:03:29.895809 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:03:29.896760 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:03:29.896760 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:29.896760 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:29.896760 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:29.896760 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:29.896760 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:29.896760 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:29.896760 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:29.896760 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:29.896760 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:29.896760 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:29.896760 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:29.896760 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:29.896760 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:29.896760 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:29.896760 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:29.896760 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:29.897658 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:03:29.897658 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:03:29.897658 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:29.897658 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:29.899346 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 349/709] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:03:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:29.908371 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dc0bf9-a974-45e3-ae7a-f3054118a017 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:29.931039 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0ee785-0a97-4279-ad54-76536b0f4f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:29.931892 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ee785-0a97-4279-ad54-76536b0f4f36 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:29.932327 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ee785-0a97-4279-ad54-76536b0f4f36 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:29.932608 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ee785-0a97-4279-ad54-76536b0f4f36 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:29.942677 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0ee785-0a97-4279-ad54-76536b0f4f36 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:29.944544 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0ee785-0a97-4279-ad54-76536b0f4f36 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:29.966428 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-1c0ee785-0a97-4279-ad54-76536b0f4f36 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:03:29.966428 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:29.966428 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:29.966428 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:29.966428 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:29.966428 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:29.966428 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:29.966428 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:29.966428 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:29.966428 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:29.966428 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:29.966428 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:29.966428 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 21:03:29.966428 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 21:03:29.966428 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 21:03:29.967566 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 21:03:29.967566 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:29.967566 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:29.967566 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:29.967566 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:29.967566 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:29.967566 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:29.967566 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:29.967566 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:29.967566 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:29.967566 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:29.967566 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:29.967566 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:29.967566 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:29.967566 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:29.967566 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:29.967566 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:29.968691 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 21:03:29.968691 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 21:03:29.968691 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 21:03:29.968691 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 21:03:29.968691 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 21:03:29.968691 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 21:03:29.968691 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:03:29.968691 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:29.975709 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 361/710] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Aug 4 21:03:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:03:29.989686 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0ee785-0a97-4279-ad54-76536b0f4f36 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:30.078229 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dc0bf9-a974-45e3-ae7a-f3054118a017 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:30.107684 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:30.113685 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceba6bdb-7e25-43ed-b224-be8e1c9118ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:30.114315 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceba6bdb-7e25-43ed-b224-be8e1c9118ba None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:30.114648 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceba6bdb-7e25-43ed-b224-be8e1c9118ba None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:30.114864 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceba6bdb-7e25-43ed-b224-be8e1c9118ba None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:30.115595 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceba6bdb-7e25-43ed-b224-be8e1c9118ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:30.119338 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceba6bdb-7e25-43ed-b224-be8e1c9118ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:30.119338 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceba6bdb-7e25-43ed-b224-be8e1c9118ba None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:30.130395 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 350/711] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:03:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:30.137466 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceba6bdb-7e25-43ed-b224-be8e1c9118ba None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:30.173727 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0ee785-0a97-4279-ad54-76536b0f4f36 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:30.217904 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c66019-df24-4686-b923-86c76b0eadee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:30.218532 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c66019-df24-4686-b923-86c76b0eadee None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:30.218812 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c66019-df24-4686-b923-86c76b0eadee None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:30.219126 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c66019-df24-4686-b923-86c76b0eadee None admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:30.220037 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c66019-df24-4686-b923-86c76b0eadee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:30.220335 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c66019-df24-4686-b923-86c76b0eadee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:30.221926 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c66019-df24-4686-b923-86c76b0eadee None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:30.236270 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 362/712] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Aug 4 21:03:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:30.237469 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f30069-4104-4903-86cc-6b8c194ebc40 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:30.237759 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f30069-4104-4903-86cc-6b8c194ebc40 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:30.238090 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f30069-4104-4903-86cc-6b8c194ebc40 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:30.243484 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 363/713] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:03:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:03:30.243484 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f30069-4104-4903-86cc-6b8c194ebc40 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:30.245472 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:30.275013 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceba6bdb-7e25-43ed-b224-be8e1c9118ba None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:30.314123 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9011bf-7584-4edb-8429-05ce3bc8b5a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:30.315073 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9011bf-7584-4edb-8429-05ce3bc8b5a4 None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:30.315601 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9011bf-7584-4edb-8429-05ce3bc8b5a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:30.316060 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9011bf-7584-4edb-8429-05ce3bc8b5a4 None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:30.316972 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9011bf-7584-4edb-8429-05ce3bc8b5a4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:30.318883 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9011bf-7584-4edb-8429-05ce3bc8b5a4 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:30.354449 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f30069-4104-4903-86cc-6b8c194ebc40 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:30.390685 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc4e2ef-8462-46dd-b904-869d27a2b956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:30.391353 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc4e2ef-8462-46dd-b904-869d27a2b956 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:30.391624 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc4e2ef-8462-46dd-b904-869d27a2b956 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:30.392025 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc4e2ef-8462-46dd-b904-869d27a2b956 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:30.393366 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc4e2ef-8462-46dd-b904-869d27a2b956 None admin] RBAC: Adding query filter params (role.id=a432dd4735944ecfab2fa5ce1705d356, scope.project.id=17dc3d39e2c048ca9070588f5717a578, user.id=6a2ce1df87fa47e5be301916bf3245fb) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:30.393673 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc4e2ef-8462-46dd-b904-869d27a2b956 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:30.395234 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc4e2ef-8462-46dd-b904-869d27a2b956 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:30.401731 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 364/714] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Aug 4 21:03:30 2022] GET /identity/v3/role_assignments?user.id=6a2ce1df87fa47e5be301916bf3245fb&scope.project.id=17dc3d39e2c048ca9070588f5717a578&role.id=a432dd4735944ecfab2fa5ce1705d356 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:30.402905 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82db4e39-79e8-4aa9-b66d-be35dafd33f1 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:30.403189 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82db4e39-79e8-4aa9-b66d-be35dafd33f1 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:30.403511 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82db4e39-79e8-4aa9-b66d-be35dafd33f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:30.431609 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 351/715] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Aug 4 21:03:30 2022] POST /identity/v3/users => generated 254 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:03:30.436847 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-82db4e39-79e8-4aa9-b66d-be35dafd33f1 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:03:30.438217 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-82db4e39-79e8-4aa9-b66d-be35dafd33f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:03:30.530339 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:30.537416 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 365/716] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:03:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:03:30.572534 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9011bf-7584-4edb-8429-05ce3bc8b5a4 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:30.627708 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:30.686679 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9011bf-7584-4edb-8429-05ce3bc8b5a4 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:30.722207 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789fd9d1-9267-4c44-9541-6dee7c0cc34f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:30.722865 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789fd9d1-9267-4c44-9541-6dee7c0cc34f None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:30.723143 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789fd9d1-9267-4c44-9541-6dee7c0cc34f None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:30.723494 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789fd9d1-9267-4c44-9541-6dee7c0cc34f None admin] PATH_INFO: `/v3/services/network` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:30.724135 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789fd9d1-9267-4c44-9541-6dee7c0cc34f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:30.725594 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789fd9d1-9267-4c44-9541-6dee7c0cc34f None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:30.738672 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-789fd9d1-9267-4c44-9541-6dee7c0cc34f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 21:03:30.738672 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:30.738672 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 21:03:30.738672 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 21:03:30.738672 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 21:03:30.738672 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 21:03:30.738672 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 21:03:30.738672 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:03:30.738672 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 21:03:30.738672 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:30.738672 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 21:03:30.738672 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:30.738672 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:30.738672 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:30.738672 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:30.738672 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:30.738672 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:30.739775 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:30.739775 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:30.739775 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:30.739775 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:30.739775 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:30.739775 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:30.739775 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 21:03:30.739775 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 21:03:30.739775 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 21:03:30.739775 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 21:03:30.739775 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:30.739775 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:30.739775 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:30.739775 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:30.739775 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:30.739775 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:30.740633 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:30.740633 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:30.740633 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:30.740633 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:30.740633 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:30.740633 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:30.740633 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:30.740633 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:30.740633 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:30.740633 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:30.740633 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 21:03:30.740633 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:03:30.740633 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 21:03:30.740633 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:30.740633 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 352/717] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Aug 4 21:03:30 2022] GET /identity/v3/services/network => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:30.745940 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82db4e39-79e8-4aa9-b66d-be35dafd33f1 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:30.854165 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82db4e39-79e8-4aa9-b66d-be35dafd33f1 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:30.890847 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adfbba6-2190-40f1-b557-f63ef5465d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:30.891464 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adfbba6-2190-40f1-b557-f63ef5465d8c None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:30.891746 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adfbba6-2190-40f1-b557-f63ef5465d8c None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:30.892020 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adfbba6-2190-40f1-b557-f63ef5465d8c None admin] PATH_INFO: `/v3/services` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:30.892764 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adfbba6-2190-40f1-b557-f63ef5465d8c None admin] RBAC: Adding query filter params (name=network) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:30.893064 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adfbba6-2190-40f1-b557-f63ef5465d8c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:30.894430 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adfbba6-2190-40f1-b557-f63ef5465d8c None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:30.899597 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 366/718] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Aug 4 21:03:30 2022] GET /identity/v3/services?name=network => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:30.904692 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789fd9d1-9267-4c44-9541-6dee7c0cc34f None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:31.048280 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789fd9d1-9267-4c44-9541-6dee7c0cc34f None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:31.062423 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:31.096585 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3939b09-f63f-4957-8d2f-1876e1fad42c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:31.097223 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3939b09-f63f-4957-8d2f-1876e1fad42c None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:31.097514 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3939b09-f63f-4957-8d2f-1876e1fad42c None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:31.097795 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3939b09-f63f-4957-8d2f-1876e1fad42c None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:31.098485 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3939b09-f63f-4957-8d2f-1876e1fad42c None admin] RBAC: Adding query filter params (type=network) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:31.098790 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3939b09-f63f-4957-8d2f-1876e1fad42c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:31.100198 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3939b09-f63f-4957-8d2f-1876e1fad42c None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:31.106809 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 353/719] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Aug 4 21:03:30 2022] GET /identity/v3/services?type=network => generated 124 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:03:31.525568 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c71e5cf-fbb7-4851-8b48-7aa4f67a04f0 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:31.526078 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c71e5cf-fbb7-4851-8b48-7aa4f67a04f0 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:31.526078 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c71e5cf-fbb7-4851-8b48-7aa4f67a04f0 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:31.527758 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 367/720] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:03:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:03:31.532685 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe56bd-9dd9-4324-b404-ef83d49e41bf None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:31.533506 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe56bd-9dd9-4324-b404-ef83d49e41bf None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:31.533782 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe56bd-9dd9-4324-b404-ef83d49e41bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:31.564380 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-61fe56bd-9dd9-4324-b404-ef83d49e41bf None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:03:31.565573 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-61fe56bd-9dd9-4324-b404-ef83d49e41bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:03:31.634572 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 354/721] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:03:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:03:31.668661 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c71e5cf-fbb7-4851-8b48-7aa4f67a04f0 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:31.743433 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:31.779403 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748055bd-f342-4d47-b109-579435071a76 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:31.779700 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748055bd-f342-4d47-b109-579435071a76 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:31.779903 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748055bd-f342-4d47-b109-579435071a76 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:31.781500 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 355/722] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:03:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:03:31.787151 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b879a1-3132-4efa-b4e2-30289655609e None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:31.787487 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b879a1-3132-4efa-b4e2-30289655609e None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:31.787846 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b879a1-3132-4efa-b4e2-30289655609e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:31.796345 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c71e5cf-fbb7-4851-8b48-7aa4f67a04f0 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:31.816861 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-d7b879a1-3132-4efa-b4e2-30289655609e None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:03:31.820332 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-d7b879a1-3132-4efa-b4e2-30289655609e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:03:31.864034 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e40daf7-a37a-4501-a5c0-92eab8f11a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:31.865073 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e40daf7-a37a-4501-a5c0-92eab8f11a03 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:31.865563 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e40daf7-a37a-4501-a5c0-92eab8f11a03 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:31.866062 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e40daf7-a37a-4501-a5c0-92eab8f11a03 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:31.884348 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e40daf7-a37a-4501-a5c0-92eab8f11a03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:03:31.890351 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-0e40daf7-a37a-4501-a5c0-92eab8f11a03 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:03:31.890351 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:31.890351 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:31.890351 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:31.890351 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:31.890351 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:31.890351 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:31.890351 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:31.890351 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:31.890351 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:31.890351 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:31.890351 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:31.890351 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:03:31.890351 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:03:31.890351 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:03:31.890351 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:03:31.891340 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:03:31.891340 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:03:31.891340 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:03:31.891340 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:03:31.891340 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:31.891340 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:31.891340 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:31.891340 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:31.891340 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:31.891340 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:31.891340 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:31.891340 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:31.891340 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:31.891340 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:31.891340 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:31.891340 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:31.892193 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:31.892193 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:31.892193 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:31.892193 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:31.892193 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:03:31.892193 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:03:31.892193 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:03:31.892193 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:03:31.892193 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:03:31.892193 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:03:31.892193 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:03:31.892193 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:31.896941 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 368/723] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Aug 4 21:03:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:31.909051 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e40daf7-a37a-4501-a5c0-92eab8f11a03 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:31.914408 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 356/724] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:03:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 21:03:31.936239 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b879a1-3132-4efa-b4e2-30289655609e None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:32.005382 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:32.027000 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e40daf7-a37a-4501-a5c0-92eab8f11a03 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:32.063306 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:32.093266 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b879a1-3132-4efa-b4e2-30289655609e None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:32.100592 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9040f8-a058-4989-a6a8-d0ea8d1bf448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:32.101489 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9040f8-a058-4989-a6a8-d0ea8d1bf448 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:32.101978 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9040f8-a058-4989-a6a8-d0ea8d1bf448 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:32.102481 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9040f8-a058-4989-a6a8-d0ea8d1bf448 None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:32.103217 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9040f8-a058-4989-a6a8-d0ea8d1bf448 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:32.103558 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9040f8-a058-4989-a6a8-d0ea8d1bf448 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:32.104939 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9040f8-a058-4989-a6a8-d0ea8d1bf448 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:32.131923 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 369/725] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Aug 4 21:03:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:32.132639 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf8ae34-940a-4831-a665-5ab467f2b416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:32.133546 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf8ae34-940a-4831-a665-5ab467f2b416 None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:32.134038 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf8ae34-940a-4831-a665-5ab467f2b416 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:32.134512 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf8ae34-940a-4831-a665-5ab467f2b416 None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:32.135444 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf8ae34-940a-4831-a665-5ab467f2b416 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:32.137281 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf8ae34-940a-4831-a665-5ab467f2b416 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:32.140308 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9040f8-a058-4989-a6a8-d0ea8d1bf448 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:32.257780 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 357/726] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Aug 4 21:03:31 2022] POST /identity/v3/services => generated 251 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:03:32.282284 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9040f8-a058-4989-a6a8-d0ea8d1bf448 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:32.335001 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d3a3b2-9214-4412-8f9f-b2af8c3f95cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:32.336119 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d3a3b2-9214-4412-8f9f-b2af8c3f95cd None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:32.336691 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d3a3b2-9214-4412-8f9f-b2af8c3f95cd None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:32.337169 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d3a3b2-9214-4412-8f9f-b2af8c3f95cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:32.348739 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d3a3b2-9214-4412-8f9f-b2af8c3f95cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:32.350537 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d3a3b2-9214-4412-8f9f-b2af8c3f95cd None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:32.367706 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-f5d3a3b2-9214-4412-8f9f-b2af8c3f95cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:32.367706 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:32.367706 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:32.367706 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:32.367706 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:32.367706 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:32.367706 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:32.367706 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:32.367706 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:32.367706 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:32.367706 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:32.367706 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:32.367706 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:03:32.367706 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:03:32.367706 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:03:32.368712 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:03:32.368712 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:32.368712 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:32.368712 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:32.368712 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:32.368712 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:32.368712 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:32.368712 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:32.368712 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:32.368712 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:32.368712 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:32.368712 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:32.368712 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:32.368712 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:32.368712 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:32.368712 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:32.368712 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:32.369684 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:03:32.369684 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:03:32.369684 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:32.369684 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:32.372123 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 370/727] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:03:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:32.378916 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf8ae34-940a-4831-a665-5ab467f2b416 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:32.506867 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:32.520064 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf8ae34-940a-4831-a665-5ab467f2b416 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:32.567982 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f4255f-0c1d-40de-9f9f-331cb50d2793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:32.568617 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f4255f-0c1d-40de-9f9f-331cb50d2793 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:32.568952 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f4255f-0c1d-40de-9f9f-331cb50d2793 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:32.569233 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f4255f-0c1d-40de-9f9f-331cb50d2793 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:32.571348 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f4255f-0c1d-40de-9f9f-331cb50d2793 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:32.571654 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f4255f-0c1d-40de-9f9f-331cb50d2793 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:32.573049 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f4255f-0c1d-40de-9f9f-331cb50d2793 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:32.586088 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 358/728] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:03:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:32.591505 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d3a3b2-9214-4412-8f9f-b2af8c3f95cd None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:32.629544 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:32.708313 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d3a3b2-9214-4412-8f9f-b2af8c3f95cd None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:32.744182 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd071ee-e4a3-4a5d-b85c-9751a1d9d5b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:32.744832 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd071ee-e4a3-4a5d-b85c-9751a1d9d5b0 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:32.745112 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd071ee-e4a3-4a5d-b85c-9751a1d9d5b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:32.745389 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd071ee-e4a3-4a5d-b85c-9751a1d9d5b0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:32.756831 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd071ee-e4a3-4a5d-b85c-9751a1d9d5b0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:32.758286 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd071ee-e4a3-4a5d-b85c-9751a1d9d5b0 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:32.770423 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-8cd071ee-e4a3-4a5d-b85c-9751a1d9d5b0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 04 21:03:32.770423 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:32.770423 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:32.770423 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:32.770423 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:32.770423 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:32.770423 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:32.770423 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:32.770423 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:32.770423 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:32.770423 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:32.770423 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:32.770423 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 21:03:32.770423 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 21:03:32.770423 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 21:03:32.770423 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 21:03:32.771423 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:32.771423 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:32.771423 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 21:03:32.771423 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:03:32.771423 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 21:03:32.771423 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:03:32.771423 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:32.771423 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:32.771423 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:32.771423 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:32.771423 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:32.771423 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:32.771423 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:32.771423 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:32.771423 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:32.771423 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:32.771423 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:32.771423 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:32.772402 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:32.772402 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:32.772402 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 21:03:32.772402 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 21:03:32.772402 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 21:03:32.772402 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 21:03:32.772402 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 21:03:32.772402 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 21:03:32.772402 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 04 21:03:32.772402 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:32.777758 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 371/729] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Aug 4 21:03:32 2022] GET /identity/v3/users/swift => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:32.778958 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f4255f-0c1d-40de-9f9f-331cb50d2793 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:32.891992 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f4255f-0c1d-40de-9f9f-331cb50d2793 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:32.926527 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1c1cf4-4d42-43d1-84f8-10192126c569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:32.927147 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1c1cf4-4d42-43d1-84f8-10192126c569 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:32.927457 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1c1cf4-4d42-43d1-84f8-10192126c569 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:32.927730 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1c1cf4-4d42-43d1-84f8-10192126c569 None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:32.928581 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1c1cf4-4d42-43d1-84f8-10192126c569 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:32.928881 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1c1cf4-4d42-43d1-84f8-10192126c569 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:32.930387 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1c1cf4-4d42-43d1-84f8-10192126c569 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:32.948792 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 359/730] 10.222.0.25 () {56 vars in 1135 bytes} [Thu Aug 4 21:03:32 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:32.956207 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd071ee-e4a3-4a5d-b85c-9751a1d9d5b0 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:32.963882 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d28b37-4c29-4107-9e42-2dc86bf453d4 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:32.964178 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d28b37-4c29-4107-9e42-2dc86bf453d4 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:32.964460 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d28b37-4c29-4107-9e42-2dc86bf453d4 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:32.965845 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 360/731] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:03:32 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:03:32.971181 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c0ed74-2559-460f-8092-7e10d6a450b5 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:32.971499 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c0ed74-2559-460f-8092-7e10d6a450b5 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:32.971781 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c0ed74-2559-460f-8092-7e10d6a450b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:33.001156 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-62c0ed74-2559-460f-8092-7e10d6a450b5 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:03:33.002918 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-62c0ed74-2559-460f-8092-7e10d6a450b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:03:33.017538 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:33.069263 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:33.078184 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd071ee-e4a3-4a5d-b85c-9751a1d9d5b0 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:33.122243 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 361/732] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:03:32 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 21:03:33.124056 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572c5765-b737-44d4-bf8e-9c0695dc3677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:33.124696 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572c5765-b737-44d4-bf8e-9c0695dc3677 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:33.124987 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572c5765-b737-44d4-bf8e-9c0695dc3677 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:33.125263 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572c5765-b737-44d4-bf8e-9c0695dc3677 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:33.157336 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa48716-a190-44ae-9934-7a18375482a0 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:33.157643 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa48716-a190-44ae-9934-7a18375482a0 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:33.158408 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa48716-a190-44ae-9934-7a18375482a0 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:33.161810 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 362/733] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:03:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:03:33.162540 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa48716-a190-44ae-9934-7a18375482a0 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:33.210404 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572c5765-b737-44d4-bf8e-9c0695dc3677 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:33.211995 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572c5765-b737-44d4-bf8e-9c0695dc3677 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:33.300119 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa48716-a190-44ae-9934-7a18375482a0 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:33.336311 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ad679e-c851-415f-b8ea-9f16f958ea7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:33.337539 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad679e-c851-415f-b8ea-9f16f958ea7e None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:33.338025 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad679e-c851-415f-b8ea-9f16f958ea7e None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:33.338624 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad679e-c851-415f-b8ea-9f16f958ea7e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:33.343021 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:33.350921 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ad679e-c851-415f-b8ea-9f16f958ea7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:03:33.355100 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 372/734] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Aug 4 21:03:32 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 21:03:33.356486 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dad9fc-792a-4eb8-88ae-c320831e5aa3 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:33.356777 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dad9fc-792a-4eb8-88ae-c320831e5aa3 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:33.357050 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dad9fc-792a-4eb8-88ae-c320831e5aa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:33.357367 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-29ad679e-c851-415f-b8ea-9f16f958ea7e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:03:33.357367 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:33.357367 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:33.357367 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:33.357367 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:33.357367 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:33.357367 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:33.357367 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:33.357367 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:33.357367 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:33.357367 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:33.357367 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:33.357367 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:03:33.357367 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:03:33.357367 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:03:33.357367 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:03:33.358511 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:03:33.358511 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:03:33.358511 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:03:33.358511 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:03:33.358511 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:33.358511 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:33.358511 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:33.358511 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:33.358511 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:33.358511 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:33.358511 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:33.358511 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:33.358511 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:33.358511 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:33.358511 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:33.358511 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:33.359375 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:33.359375 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:33.359375 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:33.359375 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:33.359375 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:03:33.359375 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:03:33.359375 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:03:33.359375 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:03:33.359375 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:03:33.359375 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:03:33.359375 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:03:33.359375 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:33.367038 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 363/735] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Aug 4 21:03:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:33.368282 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ad679e-c851-415f-b8ea-9f16f958ea7e None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:33.372451 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:33.380043 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-94dad9fc-792a-4eb8-88ae-c320831e5aa3 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:03:33.382703 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-94dad9fc-792a-4eb8-88ae-c320831e5aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:03:33.483484 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 373/736] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:03:33 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:03:33.484688 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dad9fc-792a-4eb8-88ae-c320831e5aa3 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:33.490998 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ad679e-c851-415f-b8ea-9f16f958ea7e None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:33.537644 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e3994f-aa5c-457c-990e-d2390b1cfccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:33.538761 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e3994f-aa5c-457c-990e-d2390b1cfccc None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:33.539381 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e3994f-aa5c-457c-990e-d2390b1cfccc None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:33.539856 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e3994f-aa5c-457c-990e-d2390b1cfccc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:33.551158 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e3994f-aa5c-457c-990e-d2390b1cfccc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:33.553143 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e3994f-aa5c-457c-990e-d2390b1cfccc None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:33.569732 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-e6e3994f-aa5c-457c-990e-d2390b1cfccc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:33.569732 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:33.569732 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:33.569732 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:33.569732 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:33.569732 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:33.569732 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:33.569732 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:33.569732 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:33.569732 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:33.569732 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:33.569732 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:33.569732 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:03:33.569732 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:03:33.569732 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:03:33.570677 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:03:33.570677 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:33.570677 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:33.570677 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:33.570677 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:33.570677 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:33.570677 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:33.570677 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:33.570677 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:33.570677 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:33.570677 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:33.570677 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:33.570677 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:33.570677 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:33.570677 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:33.570677 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:33.570677 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:33.571668 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:03:33.571668 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:03:33.571668 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:33.571668 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:33.573648 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 364/737] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:03:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:33.574719 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e3994f-aa5c-457c-990e-d2390b1cfccc None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:33.609601 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dad9fc-792a-4eb8-88ae-c320831e5aa3 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:33.633698 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:33.635952 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:33.646689 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d03a731-32c3-4113-9ab4-fb667a340e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:33.650532 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d03a731-32c3-4113-9ab4-fb667a340e1b None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:33.650532 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d03a731-32c3-4113-9ab4-fb667a340e1b None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:33.650532 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d03a731-32c3-4113-9ab4-fb667a340e1b None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:33.650532 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d03a731-32c3-4113-9ab4-fb667a340e1b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:33.650532 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d03a731-32c3-4113-9ab4-fb667a340e1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:33.650532 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d03a731-32c3-4113-9ab4-fb667a340e1b None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:33.664702 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 374/738] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Aug 4 21:03:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:33.664702 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d03a731-32c3-4113-9ab4-fb667a340e1b None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:33.689471 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e3994f-aa5c-457c-990e-d2390b1cfccc None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:33.729035 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082f633f-27f0-44dc-ab25-5d27802365c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:33.729682 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082f633f-27f0-44dc-ab25-5d27802365c5 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:33.729966 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082f633f-27f0-44dc-ab25-5d27802365c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:33.730243 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082f633f-27f0-44dc-ab25-5d27802365c5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:33.740670 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082f633f-27f0-44dc-ab25-5d27802365c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:03:33.746601 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-082f633f-27f0-44dc-ab25-5d27802365c5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:03:33.746601 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:33.746601 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:33.746601 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:33.746601 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:33.746601 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:33.746601 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:33.746601 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:33.746601 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:33.746601 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:33.746601 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:33.746601 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:33.746601 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:03:33.746601 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:03:33.746601 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:03:33.746601 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:03:33.747593 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:03:33.747593 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:03:33.747593 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:03:33.747593 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:03:33.747593 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:33.747593 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:33.747593 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:33.747593 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:33.747593 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:33.747593 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:33.747593 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:33.747593 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:33.747593 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:33.747593 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:33.747593 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:33.747593 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:33.748462 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:33.748462 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:33.748462 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:33.748462 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:33.748462 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:03:33.748462 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:03:33.748462 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:03:33.748462 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:03:33.748462 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:03:33.748462 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:03:33.748462 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:03:33.748462 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:33.753401 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 365/739] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Aug 4 21:03:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:33.753990 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082f633f-27f0-44dc-ab25-5d27802365c5 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:33.765426 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d03a731-32c3-4113-9ab4-fb667a340e1b None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:33.803919 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf37f235-688d-4ffc-82e4-5de6a4e03d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:33.804606 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf37f235-688d-4ffc-82e4-5de6a4e03d7b None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:33.804856 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf37f235-688d-4ffc-82e4-5de6a4e03d7b None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:33.805132 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf37f235-688d-4ffc-82e4-5de6a4e03d7b None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:33.805819 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf37f235-688d-4ffc-82e4-5de6a4e03d7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:33.806127 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf37f235-688d-4ffc-82e4-5de6a4e03d7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:33.807531 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf37f235-688d-4ffc-82e4-5de6a4e03d7b None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:33.819979 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 375/740] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:03:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:33.821114 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda6029-bd3a-4686-8a74-74d8218c51eb None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:33.821403 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda6029-bd3a-4686-8a74-74d8218c51eb None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:33.821677 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda6029-bd3a-4686-8a74-74d8218c51eb None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:33.823056 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 376/741] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:03:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:03:33.827578 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dda6029-bd3a-4686-8a74-74d8218c51eb None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:33.870913 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082f633f-27f0-44dc-ab25-5d27802365c5 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:33.881488 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:33.887706 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:33.911121 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcbbc88-2852-487b-8f3b-8cc2b22c7007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:33.911797 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcbbc88-2852-487b-8f3b-8cc2b22c7007 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:33.912083 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcbbc88-2852-487b-8f3b-8cc2b22c7007 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:33.912355 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcbbc88-2852-487b-8f3b-8cc2b22c7007 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:33.923220 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcbbc88-2852-487b-8f3b-8cc2b22c7007 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:33.924786 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcbbc88-2852-487b-8f3b-8cc2b22c7007 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:33.940800 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-7dcbbc88-2852-487b-8f3b-8cc2b22c7007 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:33.940800 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:33.940800 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:33.940800 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:33.940800 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:33.940800 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:33.940800 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:33.940800 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:33.940800 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:33.940800 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:33.940800 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:33.940800 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:33.940800 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:03:33.940800 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:03:33.940800 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:03:33.941962 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:03:33.941962 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:33.941962 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:33.941962 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:33.941962 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:33.941962 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:33.941962 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:33.941962 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:33.941962 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:33.941962 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:33.941962 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:33.941962 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:33.941962 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:33.941962 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:33.941962 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:33.941962 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:33.941962 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:33.942863 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:03:33.942863 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:03:33.942863 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:33.942863 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:33.942863 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 366/742] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:03:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:33.942863 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcbbc88-2852-487b-8f3b-8cc2b22c7007 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:33.949798 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dda6029-bd3a-4686-8a74-74d8218c51eb None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:33.987438 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6264e6-8b62-485e-adae-f5abf8600a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:33.988357 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6264e6-8b62-485e-adae-f5abf8600a48 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:33.988839 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6264e6-8b62-485e-adae-f5abf8600a48 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:33.989297 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6264e6-8b62-485e-adae-f5abf8600a48 None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:33.990203 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6264e6-8b62-485e-adae-f5abf8600a48 None admin] RBAC: Adding query filter params (name=service) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:33.991130 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6264e6-8b62-485e-adae-f5abf8600a48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:33.993081 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6264e6-8b62-485e-adae-f5abf8600a48 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:34.004268 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 377/743] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Aug 4 21:03:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:34.006601 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a9ee05-f1ad-4398-9869-f36e6c7259ed None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:34.007129 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a9ee05-f1ad-4398-9869-f36e6c7259ed None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:34.007759 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a9ee05-f1ad-4398-9869-f36e6c7259ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:34.030688 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-f0a9ee05-f1ad-4398-9869-f36e6c7259ed None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:03:34.032715 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-f0a9ee05-f1ad-4398-9869-f36e6c7259ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:03:34.053362 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcbbc88-2852-487b-8f3b-8cc2b22c7007 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:34.090606 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984363df-b19d-46e8-a749-d4a108285b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:34.091250 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984363df-b19d-46e8-a749-d4a108285b64 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:34.091561 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984363df-b19d-46e8-a749-d4a108285b64 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:34.091841 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984363df-b19d-46e8-a749-d4a108285b64 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:34.099090 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984363df-b19d-46e8-a749-d4a108285b64 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:34.100578 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984363df-b19d-46e8-a749-d4a108285b64 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:34.115343 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 378/744] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:03:34 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:03:34.115343 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a9ee05-f1ad-4398-9869-f36e6c7259ed None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:34.123672 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-984363df-b19d-46e8-a749-d4a108285b64 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:03:34.123672 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:34.123672 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:34.123672 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:34.123672 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:34.123672 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:34.123672 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:34.123672 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:34.123672 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:34.123672 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:34.123672 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:34.123672 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:34.123672 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 21:03:34.123672 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 21:03:34.123672 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 21:03:34.124858 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 21:03:34.124858 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:34.124858 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:34.124858 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:34.124858 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:34.124858 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:34.124858 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:34.124858 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:34.124858 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:34.124858 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:34.124858 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:34.124858 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:34.124858 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:34.124858 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:34.124858 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:34.124858 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:34.124858 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:34.125741 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 21:03:34.125741 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 21:03:34.125741 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 21:03:34.125741 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 21:03:34.125741 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 21:03:34.125741 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 21:03:34.125741 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:03:34.125741 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:34.125741 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 367/745] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Aug 4 21:03:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:34.131641 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984363df-b19d-46e8-a749-d4a108285b64 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:34.146883 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:34.159979 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:34.222727 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a9ee05-f1ad-4398-9869-f36e6c7259ed None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:34.243702 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984363df-b19d-46e8-a749-d4a108285b64 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:34.259709 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdece47-e4c6-4fa5-b444-625640213310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:34.260330 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdece47-e4c6-4fa5-b444-625640213310 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:34.260609 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdece47-e4c6-4fa5-b444-625640213310 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:34.260884 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdece47-e4c6-4fa5-b444-625640213310 None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:34.261573 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdece47-e4c6-4fa5-b444-625640213310 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:34.261877 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdece47-e4c6-4fa5-b444-625640213310 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:34.263253 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdece47-e4c6-4fa5-b444-625640213310 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:34.274729 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 379/746] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:03:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:34.275572 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdece47-e4c6-4fa5-b444-625640213310 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:34.293970 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d06817-6291-44af-bb95-82aac95dbe20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:34.294612 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d06817-6291-44af-bb95-82aac95dbe20 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:34.294892 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d06817-6291-44af-bb95-82aac95dbe20 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:34.295310 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d06817-6291-44af-bb95-82aac95dbe20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:34.306084 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d06817-6291-44af-bb95-82aac95dbe20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:34.307578 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d06817-6291-44af-bb95-82aac95dbe20 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:34.322911 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-86d06817-6291-44af-bb95-82aac95dbe20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:34.322911 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:34.322911 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:34.322911 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:34.322911 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:34.322911 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:34.322911 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:34.322911 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:34.322911 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:34.322911 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:34.322911 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:34.322911 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:34.322911 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:03:34.322911 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:03:34.322911 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:03:34.323924 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:03:34.323924 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:34.323924 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:34.323924 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:34.323924 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:34.323924 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:34.323924 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:34.323924 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:34.323924 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:34.323924 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:34.323924 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:34.323924 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:34.323924 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:34.323924 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:34.323924 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:34.323924 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:34.323924 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:34.324816 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:03:34.324816 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:03:34.324816 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:34.324816 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:34.324816 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 368/747] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:03:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:34.325666 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d06817-6291-44af-bb95-82aac95dbe20 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:34.379944 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:34.386542 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdece47-e4c6-4fa5-b444-625640213310 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:34.404750 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:34.424488 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583df55e-ccdc-4e81-b35f-42a30cf59bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:34.425610 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583df55e-ccdc-4e81-b35f-42a30cf59bf6 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:34.426064 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583df55e-ccdc-4e81-b35f-42a30cf59bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:34.426345 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583df55e-ccdc-4e81-b35f-42a30cf59bf6 None admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:34.427036 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d06817-6291-44af-bb95-82aac95dbe20 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:34.427164 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583df55e-ccdc-4e81-b35f-42a30cf59bf6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:34.427450 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583df55e-ccdc-4e81-b35f-42a30cf59bf6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:34.428887 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583df55e-ccdc-4e81-b35f-42a30cf59bf6 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:34.442832 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 380/748] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Aug 4 21:03:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:34.448498 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583df55e-ccdc-4e81-b35f-42a30cf59bf6 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:34.464052 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5411b7-d1bc-4af6-a7ad-affce9d34273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:34.464693 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5411b7-d1bc-4af6-a7ad-affce9d34273 None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:34.464997 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5411b7-d1bc-4af6-a7ad-affce9d34273 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:34.465476 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5411b7-d1bc-4af6-a7ad-affce9d34273 None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:34.466003 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5411b7-d1bc-4af6-a7ad-affce9d34273 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:34.467427 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5411b7-d1bc-4af6-a7ad-affce9d34273 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:34.543419 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 369/749] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Aug 4 21:03:34 2022] POST /identity/v3/services => generated 240 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:03:34.544172 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5411b7-d1bc-4af6-a7ad-affce9d34273 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:34.565716 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583df55e-ccdc-4e81-b35f-42a30cf59bf6 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:34.622153 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1912454-42a0-4e9d-99f3-c1fc381406b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:34.622839 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1912454-42a0-4e9d-99f3-c1fc381406b4 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:34.623104 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1912454-42a0-4e9d-99f3-c1fc381406b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:34.623694 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1912454-42a0-4e9d-99f3-c1fc381406b4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:34.634806 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1912454-42a0-4e9d-99f3-c1fc381406b4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:34.636613 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1912454-42a0-4e9d-99f3-c1fc381406b4 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:34.652940 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-e1912454-42a0-4e9d-99f3-c1fc381406b4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 21:03:34.652940 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:34.652940 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:34.652940 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:34.652940 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:34.652940 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:34.652940 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:34.652940 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:34.652940 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:34.652940 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:34.652940 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:34.652940 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:34.652940 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 21:03:34.652940 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 21:03:34.652940 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 21:03:34.652940 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 21:03:34.659734 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:34.659734 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:34.659734 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 21:03:34.659734 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:03:34.659734 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 21:03:34.659734 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:03:34.659734 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:34.659734 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:34.659734 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:34.659734 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:34.659734 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:34.659734 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:34.659734 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:34.659734 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:34.659734 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:34.659734 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:34.659734 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:34.659734 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:34.660808 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:34.660808 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:34.660808 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 21:03:34.660808 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 21:03:34.660808 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 21:03:34.660808 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 21:03:34.660808 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 21:03:34.660808 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 21:03:34.660808 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 21:03:34.660808 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:34.660808 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 381/750] 10.222.0.25 () {56 vars in 1093 bytes} [Thu Aug 4 21:03:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:34.660808 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1912454-42a0-4e9d-99f3-c1fc381406b4 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:34.677393 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:34.691841 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5411b7-d1bc-4af6-a7ad-affce9d34273 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:34.739972 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a5d243-d2fb-4c66-847b-a37af6bd3f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:34.740612 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a5d243-d2fb-4c66-847b-a37af6bd3f42 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:34.740898 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a5d243-d2fb-4c66-847b-a37af6bd3f42 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:34.741232 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a5d243-d2fb-4c66-847b-a37af6bd3f42 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:34.741937 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a5d243-d2fb-4c66-847b-a37af6bd3f42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:34.742242 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a5d243-d2fb-4c66-847b-a37af6bd3f42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:34.743663 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a5d243-d2fb-4c66-847b-a37af6bd3f42 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:34.752419 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:34.763371 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 370/751] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:03:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:34.763371 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a5d243-d2fb-4c66-847b-a37af6bd3f42 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:34.785048 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1912454-42a0-4e9d-99f3-c1fc381406b4 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:34.827354 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8e7af3-e57e-4c19-9679-db4984f9bb81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:34.827900 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8e7af3-e57e-4c19-9679-db4984f9bb81 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:34.828228 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8e7af3-e57e-4c19-9679-db4984f9bb81 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:34.828504 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8e7af3-e57e-4c19-9679-db4984f9bb81 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:34.829294 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8e7af3-e57e-4c19-9679-db4984f9bb81 None admin] RBAC: Adding query filter params (role.id=a432dd4735944ecfab2fa5ce1705d356, scope.project.id=17dc3d39e2c048ca9070588f5717a578, user.id=58935367c0ad445f8477cdac0df04d0a) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:34.829600 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8e7af3-e57e-4c19-9679-db4984f9bb81 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:34.831054 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8e7af3-e57e-4c19-9679-db4984f9bb81 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:34.837940 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 382/752] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Aug 4 21:03:34 2022] GET /identity/v3/role_assignments?user.id=58935367c0ad445f8477cdac0df04d0a&scope.project.id=17dc3d39e2c048ca9070588f5717a578&role.id=a432dd4735944ecfab2fa5ce1705d356 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:34.839085 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8e7af3-e57e-4c19-9679-db4984f9bb81 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:34.889584 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a5d243-d2fb-4c66-847b-a37af6bd3f42 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:34.932696 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edbbe98-38e9-46c0-b2e2-8dbf9b4cf6ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:34.933338 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edbbe98-38e9-46c0-b2e2-8dbf9b4cf6ae None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:34.933623 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edbbe98-38e9-46c0-b2e2-8dbf9b4cf6ae None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:34.933900 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edbbe98-38e9-46c0-b2e2-8dbf9b4cf6ae None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:34.934826 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edbbe98-38e9-46c0-b2e2-8dbf9b4cf6ae None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:34.935142 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edbbe98-38e9-46c0-b2e2-8dbf9b4cf6ae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:34.936787 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edbbe98-38e9-46c0-b2e2-8dbf9b4cf6ae None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:34.953498 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:34.958554 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 371/753] 10.222.0.25 () {56 vars in 1133 bytes} [Thu Aug 4 21:03:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:34.959976 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c6c6b5-e6de-4f71-99b6-cbaea563a9ba None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:34.960267 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c6c6b5-e6de-4f71-99b6-cbaea563a9ba None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:34.960546 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c6c6b5-e6de-4f71-99b6-cbaea563a9ba None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:34.965852 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 372/754] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:03:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:03:34.966768 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c6c6b5-e6de-4f71-99b6-cbaea563a9ba None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:34.989522 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8e7af3-e57e-4c19-9679-db4984f9bb81 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:35.034783 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7146d2da-2234-4df1-ace7-41eda6922a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:35.035762 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7146d2da-2234-4df1-ace7-41eda6922a58 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:35.036795 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7146d2da-2234-4df1-ace7-41eda6922a58 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:35.037242 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7146d2da-2234-4df1-ace7-41eda6922a58 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:35.040009 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:35.049102 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7146d2da-2234-4df1-ace7-41eda6922a58 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:35.050976 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7146d2da-2234-4df1-ace7-41eda6922a58 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:35.066628 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-7146d2da-2234-4df1-ace7-41eda6922a58 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 21:03:35.066628 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:35.066628 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:35.066628 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:35.066628 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:35.066628 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:35.066628 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:35.066628 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:35.066628 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:35.066628 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:35.066628 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:35.066628 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:35.066628 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 21:03:35.066628 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 21:03:35.066628 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 21:03:35.066628 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 21:03:35.067945 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:35.067945 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:35.067945 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 21:03:35.067945 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:03:35.067945 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 21:03:35.067945 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:03:35.067945 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:35.067945 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:35.067945 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:35.067945 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:35.067945 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:35.067945 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:35.067945 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:35.067945 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:35.067945 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:35.067945 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:35.067945 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:35.067945 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:35.069120 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:35.069120 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:35.069120 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 21:03:35.069120 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 21:03:35.069120 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 21:03:35.069120 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 21:03:35.069120 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 21:03:35.069120 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 21:03:35.069120 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 21:03:35.069120 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:35.071674 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 383/755] 10.222.0.25 () {56 vars in 1117 bytes} [Thu Aug 4 21:03:34 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:35.073073 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fdd199-e3c7-4862-a00e-dc920628a023 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:35.073481 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fdd199-e3c7-4862-a00e-dc920628a023 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:35.073884 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fdd199-e3c7-4862-a00e-dc920628a023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:35.097255 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-93fdd199-e3c7-4862-a00e-dc920628a023 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:03:35.098689 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-93fdd199-e3c7-4862-a00e-dc920628a023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:03:35.099881 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c6c6b5-e6de-4f71-99b6-cbaea563a9ba None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:35.148660 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c003c54-58f7-4f5b-b4af-32922fdff12e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:35.149495 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c003c54-58f7-4f5b-b4af-32922fdff12e None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:35.149876 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c003c54-58f7-4f5b-b4af-32922fdff12e None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:35.150249 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c003c54-58f7-4f5b-b4af-32922fdff12e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:35.169847 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c003c54-58f7-4f5b-b4af-32922fdff12e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:35.172118 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c003c54-58f7-4f5b-b4af-32922fdff12e None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:35.185859 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 384/756] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:03:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:03:35.186889 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fdd199-e3c7-4862-a00e-dc920628a023 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:35.193078 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-1c003c54-58f7-4f5b-b4af-32922fdff12e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:35.193078 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:35.193078 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:35.193078 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:35.193078 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:35.193078 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:35.193078 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:35.193078 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:35.193078 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:35.193078 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:35.193078 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:35.193078 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:35.193078 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:03:35.193078 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:03:35.193078 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:03:35.194104 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:03:35.194104 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:35.194104 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:35.194104 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:35.194104 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:35.194104 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:35.194104 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:35.194104 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:35.194104 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:35.194104 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:35.194104 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:35.194104 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:35.194104 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:35.194104 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:35.194104 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:35.194104 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:35.194104 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:35.195182 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:03:35.195182 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:03:35.195182 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:35.195182 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:35.196032 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 373/757] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:03:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:35.201603 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c003c54-58f7-4f5b-b4af-32922fdff12e None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:35.279470 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:35.323700 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fdd199-e3c7-4862-a00e-dc920628a023 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:35.334633 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c003c54-58f7-4f5b-b4af-32922fdff12e None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:35.359763 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:35.362671 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46959ad0-04d8-4107-b7fc-848b8677c32a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:35.363529 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46959ad0-04d8-4107-b7fc-848b8677c32a None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:35.363962 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46959ad0-04d8-4107-b7fc-848b8677c32a None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:35.364385 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46959ad0-04d8-4107-b7fc-848b8677c32a None admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:35.365333 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46959ad0-04d8-4107-b7fc-848b8677c32a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:35.365791 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46959ad0-04d8-4107-b7fc-848b8677c32a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:35.367609 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46959ad0-04d8-4107-b7fc-848b8677c32a None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:35.374543 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57bb994-863e-4393-9401-2876cfb619bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:35.375396 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57bb994-863e-4393-9401-2876cfb619bb None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:35.375727 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57bb994-863e-4393-9401-2876cfb619bb None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:35.376102 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57bb994-863e-4393-9401-2876cfb619bb None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:35.376940 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57bb994-863e-4393-9401-2876cfb619bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:35.377338 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57bb994-863e-4393-9401-2876cfb619bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:35.379595 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57bb994-863e-4393-9401-2876cfb619bb None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:35.393218 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 385/758] 10.222.0.25 () {56 vars in 1149 bytes} [Thu Aug 4 21:03:35 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:35.394415 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46959ad0-04d8-4107-b7fc-848b8677c32a None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:35.395476 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 374/759] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:03:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:35.396938 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57bb994-863e-4393-9401-2876cfb619bb None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:35.519683 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57bb994-863e-4393-9401-2876cfb619bb None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:35.522143 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46959ad0-04d8-4107-b7fc-848b8677c32a None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:35.557438 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:35.567399 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e48deae-83b3-4d7c-92eb-f68340ccaabc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:35.572594 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e48deae-83b3-4d7c-92eb-f68340ccaabc None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:35.572594 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e48deae-83b3-4d7c-92eb-f68340ccaabc None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:35.572594 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e48deae-83b3-4d7c-92eb-f68340ccaabc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:35.572836 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2517cc32-ba41-4287-986c-02a589d15eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:35.572836 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2517cc32-ba41-4287-986c-02a589d15eca None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:35.572836 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2517cc32-ba41-4287-986c-02a589d15eca None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:35.572836 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2517cc32-ba41-4287-986c-02a589d15eca None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:35.573109 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2517cc32-ba41-4287-986c-02a589d15eca None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:35.574024 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2517cc32-ba41-4287-986c-02a589d15eca None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:35.585682 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-2517cc32-ba41-4287-986c-02a589d15eca None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 21:03:35.585682 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:35.585682 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 21:03:35.585682 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 21:03:35.585682 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 21:03:35.585682 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 21:03:35.585682 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 21:03:35.585682 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:03:35.585682 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 21:03:35.585682 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:35.585682 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 21:03:35.585682 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:35.585682 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:35.585682 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:35.585682 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:35.585682 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:35.586679 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:35.586679 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:35.586679 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:35.586679 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:35.586679 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:35.586679 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:35.586679 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:35.586679 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 21:03:35.586679 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 21:03:35.586679 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 21:03:35.586679 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 21:03:35.586679 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:35.586679 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:35.586679 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:35.586679 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:35.586679 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:35.587579 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:35.587579 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:35.587579 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:35.587579 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:35.587579 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:35.587579 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:35.587579 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:35.587579 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:35.587579 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:35.587579 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:35.587579 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:35.587579 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 21:03:35.587579 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:03:35.587579 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 21:03:35.587579 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:35.591459 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 386/760] 10.222.0.25 () {56 vars in 1129 bytes} [Thu Aug 4 21:03:35 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:35.593339 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2517cc32-ba41-4287-986c-02a589d15eca None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:35.620577 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:35.665133 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e48deae-83b3-4d7c-92eb-f68340ccaabc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:35.667665 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e48deae-83b3-4d7c-92eb-f68340ccaabc None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:35.730058 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2517cc32-ba41-4287-986c-02a589d15eca None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:35.769707 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fdb676-7c7e-4b22-b44c-a735eb20e425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:35.770748 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fdb676-7c7e-4b22-b44c-a735eb20e425 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:35.771300 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fdb676-7c7e-4b22-b44c-a735eb20e425 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:35.771910 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fdb676-7c7e-4b22-b44c-a735eb20e425 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:35.781269 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fdb676-7c7e-4b22-b44c-a735eb20e425 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:35.781793 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 375/761] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Aug 4 21:03:35 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 21:03:35.782462 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e48deae-83b3-4d7c-92eb-f68340ccaabc None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:35.784500 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fdb676-7c7e-4b22-b44c-a735eb20e425 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:35.800727 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-63fdb676-7c7e-4b22-b44c-a735eb20e425 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:03:35.800727 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:35.800727 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:35.800727 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:35.800727 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:35.800727 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:35.800727 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:35.800727 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:35.800727 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:35.800727 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:35.800727 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:35.800727 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:35.800727 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 21:03:35.800727 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 21:03:35.800727 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 21:03:35.801674 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 21:03:35.801674 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:35.801674 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:35.801674 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:35.801674 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:35.801674 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:35.801674 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:35.801674 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:35.801674 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:35.801674 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:35.801674 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:35.801674 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:35.801674 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:35.801674 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:35.801674 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:35.801674 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:35.801674 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:35.802568 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 21:03:35.802568 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 21:03:35.802568 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 21:03:35.802568 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 21:03:35.802568 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 21:03:35.802568 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 21:03:35.802568 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:03:35.802568 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:35.804714 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 387/762] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Aug 4 21:03:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:35.809711 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fdb676-7c7e-4b22-b44c-a735eb20e425 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:35.863603 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:35.872714 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:35.913787 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e48deae-83b3-4d7c-92eb-f68340ccaabc None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:35.917188 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fdb676-7c7e-4b22-b44c-a735eb20e425 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:35.949659 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802b7413-318b-4420-b9ad-4eb961ba180b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:35.950614 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802b7413-318b-4420-b9ad-4eb961ba180b None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:35.951217 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802b7413-318b-4420-b9ad-4eb961ba180b None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:35.951769 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802b7413-318b-4420-b9ad-4eb961ba180b None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:35.952594 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802b7413-318b-4420-b9ad-4eb961ba180b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:35.952848 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48b7c2e-a6df-4375-91d0-3c8a3f6aca7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:35.953219 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802b7413-318b-4420-b9ad-4eb961ba180b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:35.953717 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48b7c2e-a6df-4375-91d0-3c8a3f6aca7d None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:35.954095 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48b7c2e-a6df-4375-91d0-3c8a3f6aca7d None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:35.954465 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48b7c2e-a6df-4375-91d0-3c8a3f6aca7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:35.955796 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802b7413-318b-4420-b9ad-4eb961ba180b None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:35.965562 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 376/763] 10.222.0.25 () {56 vars in 1124 bytes} [Thu Aug 4 21:03:35 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:35.965562 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802b7413-318b-4420-b9ad-4eb961ba180b None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:35.967116 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48b7c2e-a6df-4375-91d0-3c8a3f6aca7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:35.968623 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48b7c2e-a6df-4375-91d0-3c8a3f6aca7d None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:35.986062 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-c48b7c2e-a6df-4375-91d0-3c8a3f6aca7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:35.986062 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:35.986062 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:35.986062 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:35.986062 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:35.986062 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:35.986062 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:35.986062 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:35.986062 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:35.986062 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:35.986062 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:35.986062 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:35.986062 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:03:35.986062 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:03:35.986062 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:03:35.988791 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:03:35.988791 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:35.988791 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:35.988791 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:35.988791 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:35.988791 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:35.988791 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:35.988791 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:35.988791 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:35.988791 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:35.988791 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:35.988791 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:35.988791 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:35.988791 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:35.988791 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:35.988791 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:35.988791 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:35.989683 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:03:35.989683 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:03:35.989683 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:35.989683 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:35.989683 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 388/764] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:03:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:35.989683 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48b7c2e-a6df-4375-91d0-3c8a3f6aca7d None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:36.093664 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802b7413-318b-4420-b9ad-4eb961ba180b None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:36.106382 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:36.111058 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48b7c2e-a6df-4375-91d0-3c8a3f6aca7d None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:36.124845 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:36.130509 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6311eb-8ae5-429c-ac8a-5bed67ca4535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:36.131193 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6311eb-8ae5-429c-ac8a-5bed67ca4535 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:36.131520 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6311eb-8ae5-429c-ac8a-5bed67ca4535 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:36.131793 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6311eb-8ae5-429c-ac8a-5bed67ca4535 None admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:36.132539 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6311eb-8ae5-429c-ac8a-5bed67ca4535 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:36.132843 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6311eb-8ae5-429c-ac8a-5bed67ca4535 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:36.134286 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6311eb-8ae5-429c-ac8a-5bed67ca4535 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:36.149190 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 377/765] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Aug 4 21:03:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:36.153185 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6311eb-8ae5-429c-ac8a-5bed67ca4535 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:36.155234 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d12aea8-4554-4d0b-9ddd-2f5656f68d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:36.156132 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d12aea8-4554-4d0b-9ddd-2f5656f68d4f None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:36.156591 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d12aea8-4554-4d0b-9ddd-2f5656f68d4f None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:36.157041 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d12aea8-4554-4d0b-9ddd-2f5656f68d4f None admin] PATH_INFO: `/v3/services` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:36.157934 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d12aea8-4554-4d0b-9ddd-2f5656f68d4f None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:36.158842 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d12aea8-4554-4d0b-9ddd-2f5656f68d4f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:36.161119 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d12aea8-4554-4d0b-9ddd-2f5656f68d4f None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:36.168403 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 389/766] 10.222.0.25 () {56 vars in 1124 bytes} [Thu Aug 4 21:03:35 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:36.169791 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d12aea8-4554-4d0b-9ddd-2f5656f68d4f None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:36.270719 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6311eb-8ae5-429c-ac8a-5bed67ca4535 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:36.286796 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d12aea8-4554-4d0b-9ddd-2f5656f68d4f None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:36.315002 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833e3996-22d8-4ad0-9e1b-fa2da1254623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:36.316514 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833e3996-22d8-4ad0-9e1b-fa2da1254623 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:36.316995 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833e3996-22d8-4ad0-9e1b-fa2da1254623 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:36.317584 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833e3996-22d8-4ad0-9e1b-fa2da1254623 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:36.318775 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833e3996-22d8-4ad0-9e1b-fa2da1254623 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:36.319532 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833e3996-22d8-4ad0-9e1b-fa2da1254623 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:36.321097 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833e3996-22d8-4ad0-9e1b-fa2da1254623 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:36.323757 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b49186-e155-4d0a-8589-70de704329c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:36.324675 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b49186-e155-4d0a-8589-70de704329c2 None admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:36.325128 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b49186-e155-4d0a-8589-70de704329c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:36.325551 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b49186-e155-4d0a-8589-70de704329c2 None admin] PATH_INFO: `/v3/projects/17dc3d39e2c048ca9070588f5717a578/users/6a2ce1df87fa47e5be301916bf3245fb/roles/a432dd4735944ecfab2fa5ce1705d356` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:36.327697 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b49186-e155-4d0a-8589-70de704329c2 None admin] RBAC: Authorizing `identity:create_grant(project_id=17dc3d39e2c048ca9070588f5717a578, user_id=6a2ce1df87fa47e5be301916bf3245fb, role_id=a432dd4735944ecfab2fa5ce1705d356)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:36.329330 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b49186-e155-4d0a-8589-70de704329c2 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:36.334435 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 378/767] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:03:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:36.339406 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833e3996-22d8-4ad0-9e1b-fa2da1254623 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:36.388730 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:36.400315 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 390/768] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Aug 4 21:03:36 2022] PUT /identity/v3/projects/17dc3d39e2c048ca9070588f5717a578/users/6a2ce1df87fa47e5be301916bf3245fb/roles/a432dd4735944ecfab2fa5ce1705d356 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:03:36.401587 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b49186-e155-4d0a-8589-70de704329c2 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:36.433810 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:36.438893 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833e3996-22d8-4ad0-9e1b-fa2da1254623 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:36.477461 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19807fe1-811d-4ba7-8d12-742b056b9ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:36.478439 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19807fe1-811d-4ba7-8d12-742b056b9ac2 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:36.478729 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19807fe1-811d-4ba7-8d12-742b056b9ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:36.478986 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19807fe1-811d-4ba7-8d12-742b056b9ac2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:36.479795 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19807fe1-811d-4ba7-8d12-742b056b9ac2 None admin] RBAC: Adding query filter params (interface=public, service_id=9c63210d9a3d41f7a7afdf992f557b88, region_id=RegionOne) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:36.480108 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19807fe1-811d-4ba7-8d12-742b056b9ac2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:36.481810 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19807fe1-811d-4ba7-8d12-742b056b9ac2 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:36.495772 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 379/769] 10.222.0.25 () {56 vars in 1251 bytes} [Thu Aug 4 21:03:36 2022] GET /identity/v3/endpoints?service_id=9c63210d9a3d41f7a7afdf992f557b88&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:36.504191 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19807fe1-811d-4ba7-8d12-742b056b9ac2 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:36.572057 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b49186-e155-4d0a-8589-70de704329c2 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:36.633813 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdebde5-15d2-4626-9996-6010709a3602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:36.634892 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdebde5-15d2-4626-9996-6010709a3602 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:36.635445 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdebde5-15d2-4626-9996-6010709a3602 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:36.636045 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdebde5-15d2-4626-9996-6010709a3602 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:36.644843 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdebde5-15d2-4626-9996-6010709a3602 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:36.646717 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdebde5-15d2-4626-9996-6010709a3602 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:36.662240 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-ebdebde5-15d2-4626-9996-6010709a3602 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:03:36.662240 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:36.662240 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:36.662240 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:36.662240 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:36.662240 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:36.662240 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:36.662240 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:36.662240 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:36.662240 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:36.662240 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:36.662240 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:36.662240 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 21:03:36.662240 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 21:03:36.662240 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 21:03:36.663172 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 21:03:36.663172 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:36.663172 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:36.663172 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:36.663172 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:36.663172 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:36.663172 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:36.663172 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:36.663172 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:36.663172 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:36.663172 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:36.663172 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:36.663172 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:36.663172 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:36.663172 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:36.663172 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:36.663172 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:36.664110 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 21:03:36.664110 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 21:03:36.664110 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 21:03:36.664110 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 21:03:36.664110 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 21:03:36.664110 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 21:03:36.664110 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:03:36.664110 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:36.666736 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 391/770] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Aug 4 21:03:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:36.671869 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19807fe1-811d-4ba7-8d12-742b056b9ac2 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:36.672711 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdebde5-15d2-4626-9996-6010709a3602 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:36.696134 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:36.727702 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8382168c-b999-4f60-838d-ae3000a08c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:36.728333 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8382168c-b999-4f60-838d-ae3000a08c7c None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:36.728611 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8382168c-b999-4f60-838d-ae3000a08c7c None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:36.728883 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8382168c-b999-4f60-838d-ae3000a08c7c None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:36.729533 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8382168c-b999-4f60-838d-ae3000a08c7c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:36.733959 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8382168c-b999-4f60-838d-ae3000a08c7c None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:36.744434 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 380/771] 10.222.0.25 () {56 vars in 1087 bytes} [Thu Aug 4 21:03:36 2022] GET /identity/v3/services => generated 772 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:36.745698 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054077f0-a0fa-49ad-8c22-c59ba23cb041 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:36.745982 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054077f0-a0fa-49ad-8c22-c59ba23cb041 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:36.746315 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054077f0-a0fa-49ad-8c22-c59ba23cb041 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:36.747856 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 381/772] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:03:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:03:36.753618 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aee2660-5ff5-4bd3-a12e-12887c8351b3 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:36.753915 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aee2660-5ff5-4bd3-a12e-12887c8351b3 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:36.754170 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aee2660-5ff5-4bd3-a12e-12887c8351b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:36.773858 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-3aee2660-5ff5-4bd3-a12e-12887c8351b3 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:03:36.775122 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-3aee2660-5ff5-4bd3-a12e-12887c8351b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:03:36.787540 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:36.804252 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdebde5-15d2-4626-9996-6010709a3602 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:36.841057 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f25aae-b03b-4c05-aa8f-e48f7f28afc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:36.841697 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f25aae-b03b-4c05-aa8f-e48f7f28afc2 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:36.841978 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f25aae-b03b-4c05-aa8f-e48f7f28afc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:36.842251 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f25aae-b03b-4c05-aa8f-e48f7f28afc2 None admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:36.843024 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f25aae-b03b-4c05-aa8f-e48f7f28afc2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:36.843344 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f25aae-b03b-4c05-aa8f-e48f7f28afc2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:36.844848 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f25aae-b03b-4c05-aa8f-e48f7f28afc2 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:36.855004 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 382/773] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:03:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 21:03:36.858652 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 392/774] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Aug 4 21:03:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:03:36.864599 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aee2660-5ff5-4bd3-a12e-12887c8351b3 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:36.917020 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f25aae-b03b-4c05-aa8f-e48f7f28afc2 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:36.998562 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aee2660-5ff5-4bd3-a12e-12887c8351b3 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:37.011351 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:37.042314 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276aef2b-4e61-48bf-96f6-0c61a804e8d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:37.042950 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276aef2b-4e61-48bf-96f6-0c61a804e8d0 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:37.043239 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276aef2b-4e61-48bf-96f6-0c61a804e8d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:37.043702 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276aef2b-4e61-48bf-96f6-0c61a804e8d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:37.044880 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276aef2b-4e61-48bf-96f6-0c61a804e8d0 None admin] RBAC: Adding query filter params (role.id=4eb30338d61e47808b51e050836643ea, scope.project.id=17dc3d39e2c048ca9070588f5717a578, user.id=23ea380385f649628cc27e3603bc700c) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:37.045201 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276aef2b-4e61-48bf-96f6-0c61a804e8d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:37.047103 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276aef2b-4e61-48bf-96f6-0c61a804e8d0 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:37.055178 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 383/775] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Aug 4 21:03:36 2022] GET /identity/v3/role_assignments?user.id=23ea380385f649628cc27e3603bc700c&scope.project.id=17dc3d39e2c048ca9070588f5717a578&role.id=4eb30338d61e47808b51e050836643ea => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:37.056452 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00e0549-8fea-4869-846d-16b7555eb96f None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:37.056742 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00e0549-8fea-4869-846d-16b7555eb96f None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:37.057019 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00e0549-8fea-4869-846d-16b7555eb96f None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:37.058458 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 384/776] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:03:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:03:37.065427 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90319aee-0da4-4083-982f-9055ee4af073 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:37.065746 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90319aee-0da4-4083-982f-9055ee4af073 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:37.065986 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90319aee-0da4-4083-982f-9055ee4af073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:37.077024 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f25aae-b03b-4c05-aa8f-e48f7f28afc2 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:37.091268 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-90319aee-0da4-4083-982f-9055ee4af073 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:03:37.093057 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-90319aee-0da4-4083-982f-9055ee4af073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:03:37.135678 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ff5751-0d2c-49ad-8770-94768fb76bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:37.137619 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ff5751-0d2c-49ad-8770-94768fb76bf7 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:37.139308 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ff5751-0d2c-49ad-8770-94768fb76bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:37.139608 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ff5751-0d2c-49ad-8770-94768fb76bf7 None admin] PATH_INFO: `/v3/services/network` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:37.140306 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ff5751-0d2c-49ad-8770-94768fb76bf7 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:37.141719 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ff5751-0d2c-49ad-8770-94768fb76bf7 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:37.159152 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-b6ff5751-0d2c-49ad-8770-94768fb76bf7 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 21:03:37.159152 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:37.159152 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 21:03:37.159152 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 21:03:37.159152 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 21:03:37.159152 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 21:03:37.159152 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 21:03:37.159152 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:03:37.159152 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 21:03:37.159152 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:37.159152 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 21:03:37.159152 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:37.159152 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:37.159152 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:37.159152 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:37.159152 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:37.159152 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:37.160285 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:37.160285 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:37.160285 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:37.160285 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:37.160285 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:37.160285 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:37.160285 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 21:03:37.160285 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 21:03:37.160285 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 21:03:37.160285 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 21:03:37.160285 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:37.160285 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:37.160285 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:37.160285 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:37.160285 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:37.160285 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:37.161164 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:37.161164 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:37.161164 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:37.161164 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:37.161164 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:37.161164 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:37.161164 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:37.161164 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:37.161164 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:37.161164 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:37.161164 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 21:03:37.161164 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:03:37.161164 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 21:03:37.161164 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:37.164383 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 393/777] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Aug 4 21:03:36 2022] GET /identity/v3/services/network => generated 88 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:37.172412 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ff5751-0d2c-49ad-8770-94768fb76bf7 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:37.190644 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:37.199606 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 385/778] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:03:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:03:37.221885 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90319aee-0da4-4083-982f-9055ee4af073 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:37.384921 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ff5751-0d2c-49ad-8770-94768fb76bf7 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:37.405443 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90319aee-0da4-4083-982f-9055ee4af073 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:37.408306 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:37.442886 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3973add-068f-47ef-a6e9-2c8f0d9fca17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:37.443491 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3973add-068f-47ef-a6e9-2c8f0d9fca17 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:37.443827 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3973add-068f-47ef-a6e9-2c8f0d9fca17 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:37.444100 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3973add-068f-47ef-a6e9-2c8f0d9fca17 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:37.456088 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3973add-068f-47ef-a6e9-2c8f0d9fca17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:03:37.460982 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-d3973add-068f-47ef-a6e9-2c8f0d9fca17 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:03:37.460982 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:37.460982 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:37.460982 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:37.460982 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:37.460982 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:37.460982 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:37.460982 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:37.460982 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:37.460982 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:37.460982 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:37.460982 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:37.460982 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:03:37.460982 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:03:37.460982 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:03:37.460982 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:03:37.461978 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:03:37.461978 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:03:37.461978 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:03:37.461978 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:03:37.461978 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:37.461978 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:37.461978 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:37.461978 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:37.461978 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:37.461978 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:37.461978 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:37.461978 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:37.461978 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:37.461978 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:37.461978 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:37.461978 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:37.462834 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:37.462834 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:37.462834 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:37.462834 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:37.462834 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:03:37.462834 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:03:37.462834 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:03:37.462834 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:03:37.462834 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:03:37.462834 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:03:37.462834 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:03:37.462834 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:37.462834 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 386/779] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Aug 4 21:03:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:37.468309 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3973add-068f-47ef-a6e9-2c8f0d9fca17 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:37.484042 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f2a64d-e07a-4ad7-b843-bc5ea53d27f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:37.484691 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f2a64d-e07a-4ad7-b843-bc5ea53d27f0 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:37.484976 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f2a64d-e07a-4ad7-b843-bc5ea53d27f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:37.485257 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f2a64d-e07a-4ad7-b843-bc5ea53d27f0 None admin] PATH_INFO: `/v3/services` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:37.485980 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f2a64d-e07a-4ad7-b843-bc5ea53d27f0 None admin] RBAC: Adding query filter params (name=network) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:37.486291 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f2a64d-e07a-4ad7-b843-bc5ea53d27f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:37.487708 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f2a64d-e07a-4ad7-b843-bc5ea53d27f0 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:37.505448 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 394/780] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Aug 4 21:03:37 2022] GET /identity/v3/services?name=network => generated 124 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:37.520969 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f2a64d-e07a-4ad7-b843-bc5ea53d27f0 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:37.577077 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3973add-068f-47ef-a6e9-2c8f0d9fca17 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:37.611960 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cefb01a-b512-40ab-add6-f4fba7505dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:37.612579 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cefb01a-b512-40ab-add6-f4fba7505dcf None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:37.612877 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cefb01a-b512-40ab-add6-f4fba7505dcf None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:37.613197 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cefb01a-b512-40ab-add6-f4fba7505dcf None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:37.613943 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cefb01a-b512-40ab-add6-f4fba7505dcf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:37.614315 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cefb01a-b512-40ab-add6-f4fba7505dcf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:37.615740 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cefb01a-b512-40ab-add6-f4fba7505dcf None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:37.624603 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 387/781] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Aug 4 21:03:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:37.630661 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cefb01a-b512-40ab-add6-f4fba7505dcf None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:37.659810 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:37.697345 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:37.736737 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cefb01a-b512-40ab-add6-f4fba7505dcf None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:37.748748 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f2a64d-e07a-4ad7-b843-bc5ea53d27f0 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:37.772642 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9001782c-aee5-487c-b99a-440984a0b436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:37.773622 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9001782c-aee5-487c-b99a-440984a0b436 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:37.774100 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9001782c-aee5-487c-b99a-440984a0b436 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:37.774592 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9001782c-aee5-487c-b99a-440984a0b436 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:37.785824 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9001782c-aee5-487c-b99a-440984a0b436 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:37.787662 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9001782c-aee5-487c-b99a-440984a0b436 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:37.792863 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71385749-f13a-4a06-b2c6-34a6d9fe7edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:37.793564 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71385749-f13a-4a06-b2c6-34a6d9fe7edc None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:37.793856 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71385749-f13a-4a06-b2c6-34a6d9fe7edc None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:37.794136 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71385749-f13a-4a06-b2c6-34a6d9fe7edc None admin] PATH_INFO: `/v3/services` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:37.794839 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71385749-f13a-4a06-b2c6-34a6d9fe7edc None admin] RBAC: Adding query filter params (type=network) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:37.795146 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71385749-f13a-4a06-b2c6-34a6d9fe7edc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:37.796611 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71385749-f13a-4a06-b2c6-34a6d9fe7edc None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:37.802491 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 395/782] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Aug 4 21:03:37 2022] GET /identity/v3/services?type=network => generated 350 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:37.809831 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-9001782c-aee5-487c-b99a-440984a0b436 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:37.809831 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:37.809831 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:37.809831 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:37.809831 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:37.809831 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:37.809831 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:37.809831 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:37.809831 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:37.809831 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:37.809831 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:37.809831 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:37.809831 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:03:37.809831 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:03:37.809831 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:03:37.810694 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:03:37.810694 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:37.810694 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:37.810694 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:37.810694 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:37.810694 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:37.810694 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:37.810694 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:37.810694 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:37.810694 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:37.810694 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:37.810694 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:37.810694 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:37.810694 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:37.810694 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:37.810694 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:37.810694 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:37.811615 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:03:37.811615 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:03:37.811615 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:37.811615 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:37.813331 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 388/783] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:03:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:03:37.816543 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71385749-f13a-4a06-b2c6-34a6d9fe7edc None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:37.820132 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9001782c-aee5-487c-b99a-440984a0b436 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:37.899798 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:37.921970 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9001782c-aee5-487c-b99a-440984a0b436 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:37.963395 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390a81c3-2478-4751-948f-332ec0ce7db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:37.964466 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390a81c3-2478-4751-948f-332ec0ce7db1 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:37.965171 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390a81c3-2478-4751-948f-332ec0ce7db1 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:37.966828 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390a81c3-2478-4751-948f-332ec0ce7db1 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:37.966828 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390a81c3-2478-4751-948f-332ec0ce7db1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:37.966828 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390a81c3-2478-4751-948f-332ec0ce7db1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:37.968144 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390a81c3-2478-4751-948f-332ec0ce7db1 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:37.983347 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 389/784] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:03:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:03:37.989310 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390a81c3-2478-4751-948f-332ec0ce7db1 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:37.995513 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71385749-f13a-4a06-b2c6-34a6d9fe7edc None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:38.071440 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a087fe9-7c5e-4074-92cb-3726ce686b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:38.072071 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a087fe9-7c5e-4074-92cb-3726ce686b23 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:38.072400 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a087fe9-7c5e-4074-92cb-3726ce686b23 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:38.072656 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a087fe9-7c5e-4074-92cb-3726ce686b23 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:38.073458 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a087fe9-7c5e-4074-92cb-3726ce686b23 None admin] RBAC: Adding query filter params (interface=public, service_id=240a371808f94b2e927a8bd87ea1417b, region_id=RegionOne) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:38.073764 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a087fe9-7c5e-4074-92cb-3726ce686b23 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:38.076988 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a087fe9-7c5e-4074-92cb-3726ce686b23 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:38.086732 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:38.094684 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 396/785] 10.222.0.25 () {56 vars in 1251 bytes} [Thu Aug 4 21:03:37 2022] GET /identity/v3/endpoints?service_id=240a371808f94b2e927a8bd87ea1417b&interface=public®ion_id=RegionOne => generated 194 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:38.104672 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a087fe9-7c5e-4074-92cb-3726ce686b23 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:38.148389 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390a81c3-2478-4751-948f-332ec0ce7db1 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:38.193653 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b7e87f-69f8-4875-96cc-d0d522ad2372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:38.194301 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7e87f-69f8-4875-96cc-d0d522ad2372 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:38.194583 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7e87f-69f8-4875-96cc-d0d522ad2372 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:38.194852 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7e87f-69f8-4875-96cc-d0d522ad2372 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:38.208611 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b7e87f-69f8-4875-96cc-d0d522ad2372 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:38.211991 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b7e87f-69f8-4875-96cc-d0d522ad2372 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:38.216289 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:38.237058 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-46b7e87f-69f8-4875-96cc-d0d522ad2372 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 04 21:03:38.237058 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:38.237058 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:38.237058 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:38.237058 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:38.237058 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:38.237058 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:38.237058 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:38.237058 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:38.237058 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:38.237058 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:38.237058 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:38.237058 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 21:03:38.237058 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 21:03:38.237058 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 21:03:38.237058 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 21:03:38.238085 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:38.238085 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:38.238085 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 21:03:38.238085 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:03:38.238085 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 21:03:38.238085 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:03:38.238085 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:38.238085 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:38.238085 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:38.238085 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:38.238085 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:38.238085 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:38.238085 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:38.238085 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:38.238085 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:38.238085 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:38.238085 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:38.238085 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:38.239044 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:38.239044 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:38.239044 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 21:03:38.239044 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 21:03:38.239044 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 21:03:38.239044 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 21:03:38.239044 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 21:03:38.239044 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 21:03:38.239044 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 04 21:03:38.239044 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:38.244387 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 390/786] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Aug 4 21:03:37 2022] GET /identity/v3/users/swift => generated 83 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:38.249907 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b7e87f-69f8-4875-96cc-d0d522ad2372 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:38.273722 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a087fe9-7c5e-4074-92cb-3726ce686b23 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:38.318447 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6099acdb-1d57-4eda-bd0d-9601043d9390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:38.319717 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6099acdb-1d57-4eda-bd0d-9601043d9390 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:38.320190 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6099acdb-1d57-4eda-bd0d-9601043d9390 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:38.320681 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6099acdb-1d57-4eda-bd0d-9601043d9390 None admin] PATH_INFO: `/v3/services` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:38.321542 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6099acdb-1d57-4eda-bd0d-9601043d9390 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:38.323300 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6099acdb-1d57-4eda-bd0d-9601043d9390 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:38.330522 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 397/787] 10.222.0.25 () {56 vars in 1087 bytes} [Thu Aug 4 21:03:38 2022] GET /identity/v3/services => generated 772 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:03:38.462019 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b7e87f-69f8-4875-96cc-d0d522ad2372 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:38.510283 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187d5ff6-ddae-4a82-97ea-d2355ea455de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:38.511194 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187d5ff6-ddae-4a82-97ea-d2355ea455de None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:38.511729 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187d5ff6-ddae-4a82-97ea-d2355ea455de None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:38.512199 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187d5ff6-ddae-4a82-97ea-d2355ea455de None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:38.513178 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187d5ff6-ddae-4a82-97ea-d2355ea455de None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:38.513679 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187d5ff6-ddae-4a82-97ea-d2355ea455de None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:38.515300 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187d5ff6-ddae-4a82-97ea-d2355ea455de None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:38.540730 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 391/788] 10.222.0.25 () {56 vars in 1135 bytes} [Thu Aug 4 21:03:38 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:38.545963 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6099acdb-1d57-4eda-bd0d-9601043d9390 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:38.628480 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:38.677488 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6099acdb-1d57-4eda-bd0d-9601043d9390 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:38.718771 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd51c320-21e2-4d97-a1c3-a678f1bccdea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:38.719466 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd51c320-21e2-4d97-a1c3-a678f1bccdea None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:38.719749 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd51c320-21e2-4d97-a1c3-a678f1bccdea None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:38.720022 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd51c320-21e2-4d97-a1c3-a678f1bccdea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:38.731680 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd51c320-21e2-4d97-a1c3-a678f1bccdea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:38.733138 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd51c320-21e2-4d97-a1c3-a678f1bccdea None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:38.756055 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-dd51c320-21e2-4d97-a1c3-a678f1bccdea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:38.756055 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:38.756055 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:38.756055 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:38.756055 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:38.756055 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:38.756055 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:38.756055 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:38.756055 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:38.756055 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:38.756055 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:38.756055 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:38.756055 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:03:38.756055 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:03:38.756055 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:03:38.757080 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:03:38.757080 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:38.757080 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:38.757080 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:38.757080 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:38.757080 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:38.757080 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:38.757080 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:38.757080 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:38.757080 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:38.757080 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:38.757080 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:38.757080 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:38.757080 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:38.757080 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:38.757080 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:38.757080 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:38.757964 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:03:38.757964 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:03:38.757964 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:38.757964 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:38.757964 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 398/789] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:03:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:38.764408 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187d5ff6-ddae-4a82-97ea-d2355ea455de None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:38.840967 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:38.891730 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187d5ff6-ddae-4a82-97ea-d2355ea455de None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:38.937438 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf2eb8f-a96b-4fef-935d-e6bea12abd80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:38.938085 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf2eb8f-a96b-4fef-935d-e6bea12abd80 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:38.938352 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf2eb8f-a96b-4fef-935d-e6bea12abd80 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:38.938626 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf2eb8f-a96b-4fef-935d-e6bea12abd80 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:38.939381 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf2eb8f-a96b-4fef-935d-e6bea12abd80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:38.939688 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf2eb8f-a96b-4fef-935d-e6bea12abd80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:38.941133 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf2eb8f-a96b-4fef-935d-e6bea12abd80 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:38.958546 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 392/790] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:03:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:38.964361 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd51c320-21e2-4d97-a1c3-a678f1bccdea None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:39.111790 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd51c320-21e2-4d97-a1c3-a678f1bccdea None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:39.136847 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:39.166518 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1c5e12-6898-4614-9b83-56d714876b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:39.167185 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1c5e12-6898-4614-9b83-56d714876b83 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:39.167499 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1c5e12-6898-4614-9b83-56d714876b83 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:39.167781 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1c5e12-6898-4614-9b83-56d714876b83 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:39.180108 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1c5e12-6898-4614-9b83-56d714876b83 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:39.181572 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1c5e12-6898-4614-9b83-56d714876b83 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:39.193426 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-3e1c5e12-6898-4614-9b83-56d714876b83 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:03:39.193426 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:39.193426 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:39.193426 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:39.193426 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:39.193426 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:39.193426 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:39.193426 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:39.193426 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:39.193426 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:39.193426 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:39.193426 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:39.193426 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 21:03:39.193426 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 21:03:39.193426 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 21:03:39.195465 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 21:03:39.195465 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:39.195465 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:39.195465 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:39.195465 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:39.195465 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:39.195465 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:39.195465 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:39.195465 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:39.195465 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:39.195465 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:39.195465 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:39.195465 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:39.195465 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:39.195465 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:39.195465 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:39.195465 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:39.197505 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 21:03:39.197505 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 21:03:39.197505 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 21:03:39.197505 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 21:03:39.197505 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 21:03:39.197505 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 21:03:39.197505 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:03:39.197505 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:39.197505 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 399/791] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Aug 4 21:03:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:39.205671 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf2eb8f-a96b-4fef-935d-e6bea12abd80 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:39.238165 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07daedf-1819-42f6-9898-9d878540bc50 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:39.238453 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07daedf-1819-42f6-9898-9d878540bc50 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:39.238722 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07daedf-1819-42f6-9898-9d878540bc50 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:39.240374 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 400/792] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:03:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:03:39.250715 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c6ceee-21b3-4dbc-b227-050eb1e30f11 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:39.251037 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c6ceee-21b3-4dbc-b227-050eb1e30f11 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:39.251274 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c6ceee-21b3-4dbc-b227-050eb1e30f11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:39.294608 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-14c6ceee-21b3-4dbc-b227-050eb1e30f11 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:03:39.295839 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-14c6ceee-21b3-4dbc-b227-050eb1e30f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:03:39.388663 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf2eb8f-a96b-4fef-935d-e6bea12abd80 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:39.405992 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:39.414850 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 401/793] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:03:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 21:03:39.415951 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9329db-a2ea-4782-9f89-cd889944baf6 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:39.421567 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9329db-a2ea-4782-9f89-cd889944baf6 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:39.421567 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9329db-a2ea-4782-9f89-cd889944baf6 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:39.421567 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 402/794] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:03:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:03:39.421567 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15147d0e-b0ce-4821-9d16-c94ceb710c8f None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:39.421567 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15147d0e-b0ce-4821-9d16-c94ceb710c8f None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:39.421567 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15147d0e-b0ce-4821-9d16-c94ceb710c8f None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:39.421567 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 403/795] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:03:39 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:03:39.423381 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa5bf4-ebd6-4037-99cd-567e68f69c37 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:39.423812 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa5bf4-ebd6-4037-99cd-567e68f69c37 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:39.424190 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa5bf4-ebd6-4037-99cd-567e68f69c37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:39.434336 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a399b468-d684-4467-950f-a20ecc657853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:39.434989 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a399b468-d684-4467-950f-a20ecc657853 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:39.435272 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a399b468-d684-4467-950f-a20ecc657853 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:39.435576 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a399b468-d684-4467-950f-a20ecc657853 None admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:39.436322 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a399b468-d684-4467-950f-a20ecc657853 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:39.436803 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a399b468-d684-4467-950f-a20ecc657853 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:39.438312 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a399b468-d684-4467-950f-a20ecc657853 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:39.443239 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-b2fa5bf4-ebd6-4037-99cd-567e68f69c37 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:03:39.444455 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-b2fa5bf4-ebd6-4037-99cd-567e68f69c37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:03:39.451175 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 393/796] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Aug 4 21:03:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:39.452376 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a2387b-0628-43ae-b5f8-56623d7eff08 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:39.452668 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a2387b-0628-43ae-b5f8-56623d7eff08 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:39.452938 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a2387b-0628-43ae-b5f8-56623d7eff08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:39.472318 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-33a2387b-0628-43ae-b5f8-56623d7eff08 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:03:39.473518 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-33a2387b-0628-43ae-b5f8-56623d7eff08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:03:39.520808 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 404/797] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:03:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:03:39.522012 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fa5bf4-ebd6-4037-99cd-567e68f69c37 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:39.553968 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 394/798] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:03:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:03:39.553968 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a2387b-0628-43ae-b5f8-56623d7eff08 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:39.578934 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:39.646225 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fa5bf4-ebd6-4037-99cd-567e68f69c37 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:39.660759 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a2387b-0628-43ae-b5f8-56623d7eff08 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:39.669133 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:39.683353 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6653ac5-5733-4cf3-82de-031710883a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:39.684015 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6653ac5-5733-4cf3-82de-031710883a12 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:39.684299 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6653ac5-5733-4cf3-82de-031710883a12 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:39.684567 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6653ac5-5733-4cf3-82de-031710883a12 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:39.695605 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6653ac5-5733-4cf3-82de-031710883a12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:03:39.697343 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe02ccf0-dd21-4bd6-8220-73f0349e6f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:39.698018 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02ccf0-dd21-4bd6-8220-73f0349e6f1c None admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:39.698301 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02ccf0-dd21-4bd6-8220-73f0349e6f1c None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:39.698576 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02ccf0-dd21-4bd6-8220-73f0349e6f1c None admin] PATH_INFO: `/v3/projects/17dc3d39e2c048ca9070588f5717a578/users/58935367c0ad445f8477cdac0df04d0a/roles/a432dd4735944ecfab2fa5ce1705d356` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:39.700522 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-f6653ac5-5733-4cf3-82de-031710883a12 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:03:39.700522 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:39.700522 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:39.700522 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:39.700522 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:39.700522 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:39.700522 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:39.700522 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:39.700522 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:39.700522 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:39.700522 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:39.700522 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:39.700522 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:03:39.700522 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:03:39.700522 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:03:39.700522 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:03:39.701642 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:03:39.701642 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:03:39.701642 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:03:39.701642 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:03:39.701642 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:39.701642 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:39.701642 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:39.701642 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:39.701642 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:39.701642 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:39.701642 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:39.701642 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:39.701642 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:39.701642 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:39.701642 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:39.701642 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:39.708092 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:39.708092 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:39.708092 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:39.708092 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:39.708092 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:03:39.708092 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:03:39.708092 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:03:39.708092 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:03:39.708092 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:03:39.708092 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:03:39.708092 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:03:39.708092 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:39.708742 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe02ccf0-dd21-4bd6-8220-73f0349e6f1c None admin] RBAC: Authorizing `identity:create_grant(project_id=17dc3d39e2c048ca9070588f5717a578, user_id=58935367c0ad445f8477cdac0df04d0a, role_id=a432dd4735944ecfab2fa5ce1705d356)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:39.708817 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 405/799] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Aug 4 21:03:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:39.708895 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe02ccf0-dd21-4bd6-8220-73f0349e6f1c None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:39.708966 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6653ac5-5733-4cf3-82de-031710883a12 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:39.802904 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:39.806192 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 395/800] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Aug 4 21:03:39 2022] PUT /identity/v3/projects/17dc3d39e2c048ca9070588f5717a578/users/58935367c0ad445f8477cdac0df04d0a/roles/a432dd4735944ecfab2fa5ce1705d356 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:03:39.807193 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe02ccf0-dd21-4bd6-8220-73f0349e6f1c None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:39.822013 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6653ac5-5733-4cf3-82de-031710883a12 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:39.867080 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf898285-d609-408b-8809-a307c599fd07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:39.868301 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf898285-d609-408b-8809-a307c599fd07 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:39.868841 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf898285-d609-408b-8809-a307c599fd07 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:39.869354 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf898285-d609-408b-8809-a307c599fd07 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:39.885303 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf898285-d609-408b-8809-a307c599fd07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:03:39.894341 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-cf898285-d609-408b-8809-a307c599fd07 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:03:39.894341 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:39.894341 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:39.894341 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:39.894341 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:39.894341 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:39.894341 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:39.894341 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:39.894341 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:39.894341 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:39.894341 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:39.894341 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:39.894341 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:03:39.894341 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:03:39.894341 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:03:39.894341 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:03:39.895375 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:03:39.895375 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:03:39.895375 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:03:39.895375 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:03:39.895375 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:39.895375 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:39.895375 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:39.895375 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:39.895375 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:39.895375 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:39.895375 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:39.895375 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:39.895375 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:39.895375 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:39.895375 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:39.895375 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:39.896403 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:39.896403 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:39.896403 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:39.896403 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:39.896403 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:03:39.896403 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:03:39.896403 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:03:39.896403 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:03:39.896403 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:03:39.896403 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:03:39.896403 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:03:39.896403 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:39.899711 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 406/801] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Aug 4 21:03:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:39.900929 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf898285-d609-408b-8809-a307c599fd07 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:39.953556 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe02ccf0-dd21-4bd6-8220-73f0349e6f1c None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:40.008394 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23eb2c2-0fdb-46cc-9674-444539219798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:40.009060 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23eb2c2-0fdb-46cc-9674-444539219798 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:40.009611 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23eb2c2-0fdb-46cc-9674-444539219798 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:40.009895 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23eb2c2-0fdb-46cc-9674-444539219798 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:40.011032 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23eb2c2-0fdb-46cc-9674-444539219798 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:40.013089 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23eb2c2-0fdb-46cc-9674-444539219798 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:40.026356 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-d23eb2c2-0fdb-46cc-9674-444539219798 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 21:03:40.026356 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:40.026356 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 21:03:40.026356 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 21:03:40.026356 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 21:03:40.026356 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 21:03:40.026356 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 21:03:40.026356 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:03:40.026356 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 21:03:40.026356 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:40.026356 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 21:03:40.026356 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:40.026356 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:40.026356 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:40.026356 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:40.026356 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:40.027349 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:40.027349 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:40.027349 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:40.027349 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:40.027349 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:40.027349 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:40.027349 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:40.027349 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 21:03:40.027349 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 21:03:40.027349 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 21:03:40.027349 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 21:03:40.027349 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:40.027349 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:40.027349 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:40.027349 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:40.027349 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:40.028201 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:40.028201 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:40.028201 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:40.028201 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:40.028201 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:40.028201 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:40.028201 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:40.028201 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:40.028201 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:40.028201 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:40.028201 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:40.028201 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 21:03:40.028201 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:03:40.028201 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 21:03:40.028201 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:40.029013 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 396/802] 10.222.0.25 () {56 vars in 1129 bytes} [Thu Aug 4 21:03:39 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:40.029606 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23eb2c2-0fdb-46cc-9674-444539219798 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:40.030727 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf898285-d609-408b-8809-a307c599fd07 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:40.075439 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:40.081786 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8c5df3-cfe4-4bb2-b4a9-0aa9a5c29bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:40.083308 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8c5df3-cfe4-4bb2-b4a9-0aa9a5c29bc5 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:40.083556 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8c5df3-cfe4-4bb2-b4a9-0aa9a5c29bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:40.083832 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8c5df3-cfe4-4bb2-b4a9-0aa9a5c29bc5 None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:40.084573 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8c5df3-cfe4-4bb2-b4a9-0aa9a5c29bc5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:40.084879 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8c5df3-cfe4-4bb2-b4a9-0aa9a5c29bc5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:40.088613 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8c5df3-cfe4-4bb2-b4a9-0aa9a5c29bc5 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:40.100592 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 407/803] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Aug 4 21:03:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:40.101862 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8c5df3-cfe4-4bb2-b4a9-0aa9a5c29bc5 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:40.128673 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:40.175120 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23eb2c2-0fdb-46cc-9674-444539219798 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:40.211420 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25641063-8149-4481-9cb8-0b40147015e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:40.212416 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25641063-8149-4481-9cb8-0b40147015e9 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:40.212901 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25641063-8149-4481-9cb8-0b40147015e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:40.213370 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25641063-8149-4481-9cb8-0b40147015e9 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:40.214277 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25641063-8149-4481-9cb8-0b40147015e9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:40.214791 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25641063-8149-4481-9cb8-0b40147015e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:40.216437 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25641063-8149-4481-9cb8-0b40147015e9 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:40.226122 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 397/804] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Aug 4 21:03:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:40.227361 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25641063-8149-4481-9cb8-0b40147015e9 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:40.231088 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8c5df3-cfe4-4bb2-b4a9-0aa9a5c29bc5 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:40.269896 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b921f404-441e-4731-a6d3-672802037605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:40.270716 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b921f404-441e-4731-a6d3-672802037605 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:40.270899 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b921f404-441e-4731-a6d3-672802037605 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:40.271195 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b921f404-441e-4731-a6d3-672802037605 None admin] PATH_INFO: `/v3/services` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:40.272011 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b921f404-441e-4731-a6d3-672802037605 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:40.272322 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b921f404-441e-4731-a6d3-672802037605 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:40.273708 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b921f404-441e-4731-a6d3-672802037605 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:40.283676 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 408/805] 10.222.0.25 () {56 vars in 1124 bytes} [Thu Aug 4 21:03:40 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:40.284810 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b921f404-441e-4731-a6d3-672802037605 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:40.336697 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25641063-8149-4481-9cb8-0b40147015e9 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:40.339722 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:40.373815 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7c7284-a66b-41c9-9a1d-aa9ceabb5a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:40.374125 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:40.374912 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7c7284-a66b-41c9-9a1d-aa9ceabb5a9c None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:40.375129 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7c7284-a66b-41c9-9a1d-aa9ceabb5a9c None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:40.375601 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7c7284-a66b-41c9-9a1d-aa9ceabb5a9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:40.386265 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7c7284-a66b-41c9-9a1d-aa9ceabb5a9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:40.388167 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7c7284-a66b-41c9-9a1d-aa9ceabb5a9c None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:40.395054 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b921f404-441e-4731-a6d3-672802037605 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:40.404688 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-0e7c7284-a66b-41c9-9a1d-aa9ceabb5a9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:40.404688 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:40.404688 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:40.404688 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:40.404688 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:40.404688 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:40.404688 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:40.404688 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:40.404688 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:40.404688 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:40.404688 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:40.404688 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:40.404688 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:03:40.404688 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:03:40.404688 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:03:40.406078 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:03:40.406078 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:40.406078 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:40.406078 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:40.406078 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:40.406078 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:40.406078 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:40.406078 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:40.406078 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:40.406078 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:40.406078 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:40.406078 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:40.406078 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:40.406078 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:40.406078 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:40.406078 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:40.406078 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:40.407157 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:03:40.407157 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:03:40.407157 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:40.407157 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:40.413977 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 398/806] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:03:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:40.413977 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7c7284-a66b-41c9-9a1d-aa9ceabb5a9c None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:40.440028 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83960a4-b7b3-4477-a45b-4d81e2c371fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:40.441008 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83960a4-b7b3-4477-a45b-4d81e2c371fa None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:40.441514 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83960a4-b7b3-4477-a45b-4d81e2c371fa None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:40.441991 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83960a4-b7b3-4477-a45b-4d81e2c371fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:40.453004 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83960a4-b7b3-4477-a45b-4d81e2c371fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:40.455379 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83960a4-b7b3-4477-a45b-4d81e2c371fa None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:40.471976 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-f83960a4-b7b3-4477-a45b-4d81e2c371fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:40.471976 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:40.471976 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:40.471976 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:40.471976 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:40.471976 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:40.471976 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:40.471976 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:40.471976 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:40.471976 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:40.471976 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:40.471976 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:40.471976 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:03:40.471976 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:03:40.471976 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:03:40.472905 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:03:40.472905 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:40.472905 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:40.472905 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:40.472905 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:40.472905 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:40.472905 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:40.472905 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:40.472905 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:40.472905 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:40.472905 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:40.472905 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:40.472905 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:40.472905 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:40.472905 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:40.472905 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:40.472905 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:40.473795 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:03:40.473795 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:03:40.473795 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:40.473795 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:40.480361 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 409/807] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:03:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:40.481437 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83960a4-b7b3-4477-a45b-4d81e2c371fa None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:40.518388 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7c7284-a66b-41c9-9a1d-aa9ceabb5a9c None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:40.554854 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aee2243-c9bd-4b94-bf61-328e9f782707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:40.555565 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aee2243-c9bd-4b94-bf61-328e9f782707 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:40.556519 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aee2243-c9bd-4b94-bf61-328e9f782707 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:40.556519 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aee2243-c9bd-4b94-bf61-328e9f782707 None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:40.556890 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aee2243-c9bd-4b94-bf61-328e9f782707 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:40.557193 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aee2243-c9bd-4b94-bf61-328e9f782707 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:40.558632 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aee2243-c9bd-4b94-bf61-328e9f782707 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:40.569436 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 399/808] 10.222.0.25 () {56 vars in 1124 bytes} [Thu Aug 4 21:03:40 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:40.569436 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aee2243-c9bd-4b94-bf61-328e9f782707 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:40.579732 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:40.594482 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83960a4-b7b3-4477-a45b-4d81e2c371fa None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:40.630210 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbef799-5326-4a04-ac43-6965584be64f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:40.631196 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbef799-5326-4a04-ac43-6965584be64f None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:40.631807 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbef799-5326-4a04-ac43-6965584be64f None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:40.632355 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbef799-5326-4a04-ac43-6965584be64f None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:40.641158 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbef799-5326-4a04-ac43-6965584be64f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:40.641158 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbef799-5326-4a04-ac43-6965584be64f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:40.641158 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbef799-5326-4a04-ac43-6965584be64f None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:40.641158 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:40.654860 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 410/809] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:03:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:40.656168 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbef799-5326-4a04-ac43-6965584be64f None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:40.684605 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aee2243-c9bd-4b94-bf61-328e9f782707 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:40.732274 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d070a67b-9c43-48ff-a7dd-29498ecd8154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:40.732817 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d070a67b-9c43-48ff-a7dd-29498ecd8154 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:40.733104 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d070a67b-9c43-48ff-a7dd-29498ecd8154 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:40.733380 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d070a67b-9c43-48ff-a7dd-29498ecd8154 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:40.734070 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d070a67b-9c43-48ff-a7dd-29498ecd8154 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:40.734375 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d070a67b-9c43-48ff-a7dd-29498ecd8154 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:40.736204 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d070a67b-9c43-48ff-a7dd-29498ecd8154 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:40.756268 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 400/810] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:03:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:40.756268 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d070a67b-9c43-48ff-a7dd-29498ecd8154 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:40.761113 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbef799-5326-4a04-ac43-6965584be64f None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:40.797800 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2164d799-e690-468c-90ea-90c766a1fe0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:40.798449 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2164d799-e690-468c-90ea-90c766a1fe0a None admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:40.798732 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2164d799-e690-468c-90ea-90c766a1fe0a None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:40.799004 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2164d799-e690-468c-90ea-90c766a1fe0a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:40.799743 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2164d799-e690-468c-90ea-90c766a1fe0a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:40.801120 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2164d799-e690-468c-90ea-90c766a1fe0a None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:40.823128 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:40.855969 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d070a67b-9c43-48ff-a7dd-29498ecd8154 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:40.887879 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ed04f7-dae0-49e4-bfd0-60dadde0596f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:40.888518 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ed04f7-dae0-49e4-bfd0-60dadde0596f None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:40.888802 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ed04f7-dae0-49e4-bfd0-60dadde0596f None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:40.889080 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ed04f7-dae0-49e4-bfd0-60dadde0596f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:40.893232 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 411/811] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:03:40 2022] POST /identity/v3/endpoints => generated 346 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:03:40.894638 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f281dc6-c7ea-45e9-8899-2a2b9b7cfc3c None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:40.896279 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f281dc6-c7ea-45e9-8899-2a2b9b7cfc3c None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:40.896279 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f281dc6-c7ea-45e9-8899-2a2b9b7cfc3c None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:40.897051 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 412/812] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:03:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:03:40.897962 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f281dc6-c7ea-45e9-8899-2a2b9b7cfc3c None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:40.905274 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ed04f7-dae0-49e4-bfd0-60dadde0596f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:40.906791 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ed04f7-dae0-49e4-bfd0-60dadde0596f None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:40.920290 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-71ed04f7-dae0-49e4-bfd0-60dadde0596f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 21:03:40.920290 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:40.920290 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:40.920290 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:40.920290 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:40.920290 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:40.920290 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:40.920290 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:40.920290 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:40.920290 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:40.920290 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:40.920290 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:40.920290 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 21:03:40.920290 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 21:03:40.920290 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 21:03:40.920290 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 21:03:40.921297 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:40.921297 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:40.921297 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 21:03:40.921297 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:03:40.921297 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 21:03:40.921297 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:03:40.921297 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:40.921297 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:40.921297 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:40.921297 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:40.921297 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:40.921297 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:40.921297 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:40.921297 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:40.921297 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:40.921297 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:40.921297 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:40.921297 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:40.922237 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:40.922237 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:40.922237 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 21:03:40.922237 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 21:03:40.922237 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 21:03:40.922237 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 21:03:40.922237 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 21:03:40.922237 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 21:03:40.922237 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 21:03:40.922237 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:40.922237 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 401/813] 10.222.0.25 () {56 vars in 1093 bytes} [Thu Aug 4 21:03:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:40.929945 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c670aca-34e7-4418-9316-5307568b7ae2 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:40.929945 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c670aca-34e7-4418-9316-5307568b7ae2 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:40.929945 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c670aca-34e7-4418-9316-5307568b7ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:40.946394 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-2c670aca-34e7-4418-9316-5307568b7ae2 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:03:40.947712 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-2c670aca-34e7-4418-9316-5307568b7ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:03:41.028417 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:41.049471 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 402/814] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:03:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:03:41.050881 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c670aca-34e7-4418-9316-5307568b7ae2 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:41.084132 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f281dc6-c7ea-45e9-8899-2a2b9b7cfc3c None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:41.122145 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:41.130780 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28788307-c77d-4e79-a122-2d61eb65cc4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:41.131470 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28788307-c77d-4e79-a122-2d61eb65cc4e None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:41.131759 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28788307-c77d-4e79-a122-2d61eb65cc4e None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:41.132039 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28788307-c77d-4e79-a122-2d61eb65cc4e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:41.140033 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28788307-c77d-4e79-a122-2d61eb65cc4e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:41.141577 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28788307-c77d-4e79-a122-2d61eb65cc4e None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:41.154151 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-28788307-c77d-4e79-a122-2d61eb65cc4e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 21:03:41.154151 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:41.154151 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:41.154151 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:41.154151 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:41.154151 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:41.154151 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:41.154151 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:41.154151 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:41.154151 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:41.154151 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:41.154151 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:41.154151 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 21:03:41.154151 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 21:03:41.154151 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 21:03:41.154151 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 21:03:41.155176 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:41.155176 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:41.155176 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 21:03:41.155176 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:03:41.155176 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 21:03:41.155176 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:03:41.155176 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:41.155176 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:41.155176 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:41.155176 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:41.155176 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:41.155176 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:41.155176 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:41.155176 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:41.155176 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:41.155176 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:41.155176 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:41.155176 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:41.156157 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:41.156157 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:41.156157 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 21:03:41.156157 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 21:03:41.156157 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 21:03:41.156157 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 21:03:41.156157 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 21:03:41.156157 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 21:03:41.156157 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 21:03:41.156157 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:41.156157 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 413/815] 10.222.0.25 () {56 vars in 1117 bytes} [Thu Aug 4 21:03:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:41.156744 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28788307-c77d-4e79-a122-2d61eb65cc4e None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:41.171543 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c670aca-34e7-4418-9316-5307568b7ae2 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:41.207989 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2205047-fb5b-4ae4-a65d-5014752fb076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:41.208619 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2205047-fb5b-4ae4-a65d-5014752fb076 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:41.208907 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2205047-fb5b-4ae4-a65d-5014752fb076 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:41.209195 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2205047-fb5b-4ae4-a65d-5014752fb076 None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:41.209992 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2205047-fb5b-4ae4-a65d-5014752fb076 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:41.210304 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2205047-fb5b-4ae4-a65d-5014752fb076 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:41.211788 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2205047-fb5b-4ae4-a65d-5014752fb076 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:41.239060 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 403/816] 10.222.0.25 () {56 vars in 1133 bytes} [Thu Aug 4 21:03:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:41.240177 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2205047-fb5b-4ae4-a65d-5014752fb076 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:41.290477 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28788307-c77d-4e79-a122-2d61eb65cc4e None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:41.301942 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:41.328460 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad31c9cd-2320-4155-92b7-a3025829438d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:41.329183 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad31c9cd-2320-4155-92b7-a3025829438d None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:41.329464 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad31c9cd-2320-4155-92b7-a3025829438d None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:41.329827 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad31c9cd-2320-4155-92b7-a3025829438d None admin] PATH_INFO: `/v3/services/network` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:41.330755 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad31c9cd-2320-4155-92b7-a3025829438d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:41.332034 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad31c9cd-2320-4155-92b7-a3025829438d None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:41.342776 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-ad31c9cd-2320-4155-92b7-a3025829438d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 21:03:41.342776 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:41.342776 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 21:03:41.342776 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 21:03:41.342776 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 21:03:41.342776 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 21:03:41.342776 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 21:03:41.342776 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:03:41.342776 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 21:03:41.342776 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:41.342776 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 21:03:41.342776 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:41.342776 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:41.342776 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:41.342776 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:41.342776 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:41.342776 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:41.343836 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:41.343836 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:41.343836 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:41.343836 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:41.343836 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:41.343836 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:41.343836 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 21:03:41.343836 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 21:03:41.343836 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 21:03:41.343836 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 21:03:41.343836 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:41.343836 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:41.343836 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:41.343836 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:41.343836 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:41.343836 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:41.344670 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:41.344670 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:41.344670 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:41.344670 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:41.344670 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:41.344670 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:41.344670 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:41.344670 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:41.344670 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:41.344670 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:41.344670 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 21:03:41.344670 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:03:41.344670 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 21:03:41.344670 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:41.344670 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 414/817] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Aug 4 21:03:41 2022] GET /identity/v3/services/network => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:41.348051 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad31c9cd-2320-4155-92b7-a3025829438d None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:41.357828 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2205047-fb5b-4ae4-a65d-5014752fb076 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:41.373141 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:41.400704 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6152b517-4069-41f0-9a1d-e49beacb8dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:41.401876 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6152b517-4069-41f0-9a1d-e49beacb8dd7 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:41.402314 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6152b517-4069-41f0-9a1d-e49beacb8dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:41.402955 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6152b517-4069-41f0-9a1d-e49beacb8dd7 None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:41.404068 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6152b517-4069-41f0-9a1d-e49beacb8dd7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:41.404931 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6152b517-4069-41f0-9a1d-e49beacb8dd7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:41.406970 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6152b517-4069-41f0-9a1d-e49beacb8dd7 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:41.431383 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 404/818] 10.222.0.25 () {56 vars in 1149 bytes} [Thu Aug 4 21:03:41 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:41.431383 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6152b517-4069-41f0-9a1d-e49beacb8dd7 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:41.478564 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad31c9cd-2320-4155-92b7-a3025829438d None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:41.530875 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cf2947-c03a-4b79-8f6c-064cced28d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:41.531674 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cf2947-c03a-4b79-8f6c-064cced28d00 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:41.531964 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cf2947-c03a-4b79-8f6c-064cced28d00 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:41.532316 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cf2947-c03a-4b79-8f6c-064cced28d00 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:41.541219 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6152b517-4069-41f0-9a1d-e49beacb8dd7 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:41.578108 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f838552e-1bf4-428c-a404-76fbddfb5c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:41.579083 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f838552e-1bf4-428c-a404-76fbddfb5c86 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:41.579671 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f838552e-1bf4-428c-a404-76fbddfb5c86 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:41.580205 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f838552e-1bf4-428c-a404-76fbddfb5c86 None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:41.581144 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f838552e-1bf4-428c-a404-76fbddfb5c86 None admin] RBAC: Adding query filter params (name=network) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:41.581668 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f838552e-1bf4-428c-a404-76fbddfb5c86 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:41.583308 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f838552e-1bf4-428c-a404-76fbddfb5c86 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:41.589328 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 405/819] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Aug 4 21:03:41 2022] GET /identity/v3/services?name=network => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:41.589804 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:41.591032 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f838552e-1bf4-428c-a404-76fbddfb5c86 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:41.627072 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cf2947-c03a-4b79-8f6c-064cced28d00 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:41.628594 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cf2947-c03a-4b79-8f6c-064cced28d00 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:41.629592 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:41.712925 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f838552e-1bf4-428c-a404-76fbddfb5c86 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:41.753460 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb516322-bd42-4ea0-9a3c-bd44a75b1dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:41.754520 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb516322-bd42-4ea0-9a3c-bd44a75b1dd4 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:41.755232 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb516322-bd42-4ea0-9a3c-bd44a75b1dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:41.755860 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb516322-bd42-4ea0-9a3c-bd44a75b1dd4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:41.760516 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 415/820] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Aug 4 21:03:41 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 21:03:41.761392 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cf2947-c03a-4b79-8f6c-064cced28d00 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:41.768688 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb516322-bd42-4ea0-9a3c-bd44a75b1dd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:41.770447 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb516322-bd42-4ea0-9a3c-bd44a75b1dd4 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:41.787964 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-bb516322-bd42-4ea0-9a3c-bd44a75b1dd4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:41.787964 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:41.787964 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:41.787964 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:41.787964 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:41.787964 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:41.787964 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:41.787964 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:41.787964 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:41.787964 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:41.787964 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:41.787964 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:41.787964 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:03:41.787964 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:03:41.787964 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:03:41.788942 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:03:41.788942 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:41.788942 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:41.788942 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:41.788942 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:41.788942 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:41.788942 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:41.788942 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:41.788942 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:41.788942 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:41.788942 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:41.788942 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:41.788942 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:41.788942 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:41.788942 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:41.788942 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:41.788942 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:41.789877 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:03:41.789877 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:03:41.789877 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:41.789877 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:41.792045 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 406/821] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:03:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:41.793136 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb516322-bd42-4ea0-9a3c-bd44a75b1dd4 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:41.859764 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:41.885558 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cf2947-c03a-4b79-8f6c-064cced28d00 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:41.895759 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:41.916804 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb516322-bd42-4ea0-9a3c-bd44a75b1dd4 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:41.923233 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3bd1f0-099f-4833-9f83-61be68502322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:41.923908 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3bd1f0-099f-4833-9f83-61be68502322 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:41.924192 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3bd1f0-099f-4833-9f83-61be68502322 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:41.924466 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3bd1f0-099f-4833-9f83-61be68502322 None admin] PATH_INFO: `/v3/services` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:41.925147 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3bd1f0-099f-4833-9f83-61be68502322 None admin] RBAC: Adding query filter params (type=network) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:41.925455 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3bd1f0-099f-4833-9f83-61be68502322 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:41.926826 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3bd1f0-099f-4833-9f83-61be68502322 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:41.931775 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 416/822] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Aug 4 21:03:41 2022] GET /identity/v3/services?type=network => generated 350 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:41.932602 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3bd1f0-099f-4833-9f83-61be68502322 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:41.961055 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53e8057-d9e8-43a8-9770-5239970286df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:41.961585 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53e8057-d9e8-43a8-9770-5239970286df None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:41.961868 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53e8057-d9e8-43a8-9770-5239970286df None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:41.962261 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53e8057-d9e8-43a8-9770-5239970286df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:41.973721 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53e8057-d9e8-43a8-9770-5239970286df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:41.975028 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53e8057-d9e8-43a8-9770-5239970286df None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:41.990089 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-f53e8057-d9e8-43a8-9770-5239970286df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:41.990089 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:41.990089 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:41.990089 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:41.990089 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:41.990089 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:41.990089 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:41.990089 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:41.990089 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:41.990089 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:41.990089 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:41.990089 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:41.990089 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:03:41.990089 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:03:41.990089 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:03:41.991087 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:03:41.991087 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:41.991087 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:41.991087 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:41.991087 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:41.991087 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:41.991087 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:41.991087 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:41.991087 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:41.991087 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:41.991087 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:41.991087 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:41.991087 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:41.991087 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:41.991087 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:41.991087 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:41.991087 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:41.992079 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:03:41.992079 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:03:41.992079 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:41.992079 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:41.992079 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 407/823] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:03:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:41.997013 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589d9a3b-446b-457d-83e3-4d865b9bfee5 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:41.997480 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589d9a3b-446b-457d-83e3-4d865b9bfee5 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:41.997756 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589d9a3b-446b-457d-83e3-4d865b9bfee5 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:41.999212 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 408/824] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:03:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:03:42.003354 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589d9a3b-446b-457d-83e3-4d865b9bfee5 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:42.053163 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3bd1f0-099f-4833-9f83-61be68502322 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:42.090673 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73833694-d518-4a71-bed0-aebc7582e92d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:42.091349 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73833694-d518-4a71-bed0-aebc7582e92d None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:42.092003 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73833694-d518-4a71-bed0-aebc7582e92d None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:42.092289 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73833694-d518-4a71-bed0-aebc7582e92d None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:42.093277 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73833694-d518-4a71-bed0-aebc7582e92d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:42.093588 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73833694-d518-4a71-bed0-aebc7582e92d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:42.095726 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73833694-d518-4a71-bed0-aebc7582e92d None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:42.108240 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:42.113813 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 417/825] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:03:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:42.115044 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73833694-d518-4a71-bed0-aebc7582e92d None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:42.117412 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589d9a3b-446b-457d-83e3-4d865b9bfee5 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:42.153856 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef2614a-5470-4758-b34f-8d7c356ab85c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:42.154488 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef2614a-5470-4758-b34f-8d7c356ab85c None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:42.154773 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef2614a-5470-4758-b34f-8d7c356ab85c None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:42.155049 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef2614a-5470-4758-b34f-8d7c356ab85c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:42.155809 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef2614a-5470-4758-b34f-8d7c356ab85c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:42.157206 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef2614a-5470-4758-b34f-8d7c356ab85c None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:42.166559 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:42.226831 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73833694-d518-4a71-bed0-aebc7582e92d None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:42.262102 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7159f5-7ba2-48a8-a5ac-02a0c74477b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:42.262815 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7159f5-7ba2-48a8-a5ac-02a0c74477b6 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:42.263101 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7159f5-7ba2-48a8-a5ac-02a0c74477b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:42.263444 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7159f5-7ba2-48a8-a5ac-02a0c74477b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:42.264097 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7159f5-7ba2-48a8-a5ac-02a0c74477b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:42.264406 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7159f5-7ba2-48a8-a5ac-02a0c74477b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:42.265771 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7159f5-7ba2-48a8-a5ac-02a0c74477b6 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:42.279381 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 409/826] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:03:41 2022] POST /identity/v3/endpoints => generated 337 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:03:42.279381 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be29c08-621f-4a50-85c4-cbe053aac8ce None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:42.279381 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be29c08-621f-4a50-85c4-cbe053aac8ce None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:42.279381 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be29c08-621f-4a50-85c4-cbe053aac8ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:42.279822 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 418/827] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:03:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:42.279822 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7159f5-7ba2-48a8-a5ac-02a0c74477b6 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:42.297712 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-9be29c08-621f-4a50-85c4-cbe053aac8ce None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:03:42.299113 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-9be29c08-621f-4a50-85c4-cbe053aac8ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:03:42.349669 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:42.395541 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7159f5-7ba2-48a8-a5ac-02a0c74477b6 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:42.399474 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 410/828] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:03:42 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:03:42.400464 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be29c08-621f-4a50-85c4-cbe053aac8ce None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:42.450190 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c2cb3c-c997-44e0-8008-867c09a63792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:42.450845 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c2cb3c-c997-44e0-8008-867c09a63792 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:42.451130 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c2cb3c-c997-44e0-8008-867c09a63792 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:42.451425 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c2cb3c-c997-44e0-8008-867c09a63792 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:42.459851 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c2cb3c-c997-44e0-8008-867c09a63792 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:42.461302 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c2cb3c-c997-44e0-8008-867c09a63792 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:42.475675 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-58c2cb3c-c997-44e0-8008-867c09a63792 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:03:42.475675 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:42.475675 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:42.475675 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:42.475675 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:42.475675 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:42.475675 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:42.475675 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:42.475675 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:42.475675 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:42.475675 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:42.475675 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:42.475675 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 21:03:42.475675 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 21:03:42.475675 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 21:03:42.476769 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 21:03:42.476769 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:42.476769 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:42.476769 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:42.476769 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:42.476769 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:42.476769 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:42.476769 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:42.476769 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:42.476769 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:42.476769 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:42.476769 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:42.476769 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:42.476769 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:42.476769 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:42.476769 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:42.476769 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:42.477860 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 21:03:42.477860 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 21:03:42.477860 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 21:03:42.477860 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 21:03:42.477860 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 21:03:42.477860 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 21:03:42.477860 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:03:42.477860 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:42.481727 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 419/829] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Aug 4 21:03:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:42.482606 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c2cb3c-c997-44e0-8008-867c09a63792 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:42.547535 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be29c08-621f-4a50-85c4-cbe053aac8ce None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:42.569796 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:42.583813 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7590d05-7b25-4ed3-9a9b-2a5dfba2a625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:42.584468 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7590d05-7b25-4ed3-9a9b-2a5dfba2a625 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:42.584747 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7590d05-7b25-4ed3-9a9b-2a5dfba2a625 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:42.585026 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7590d05-7b25-4ed3-9a9b-2a5dfba2a625 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:42.591885 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7590d05-7b25-4ed3-9a9b-2a5dfba2a625 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:42.593416 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7590d05-7b25-4ed3-9a9b-2a5dfba2a625 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:42.606162 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c2cb3c-c997-44e0-8008-867c09a63792 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:42.606533 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-e7590d05-7b25-4ed3-9a9b-2a5dfba2a625 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:03:42.606533 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:42.606533 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:42.606533 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:42.606533 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:42.606533 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:42.606533 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:42.606533 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:42.606533 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:42.606533 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:42.606533 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:42.606533 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:42.606533 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 21:03:42.606533 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 21:03:42.606533 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 21:03:42.607430 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 21:03:42.607430 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:42.607430 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:42.607430 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:42.607430 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:42.607430 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:42.607430 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:42.607430 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:42.607430 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:42.607430 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:42.607430 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:42.607430 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:42.607430 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:42.607430 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:42.607430 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:42.607430 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:42.607430 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:42.608319 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 21:03:42.608319 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 21:03:42.608319 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 21:03:42.608319 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 21:03:42.608319 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 21:03:42.608319 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 21:03:42.608319 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:03:42.608319 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:42.608319 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 411/830] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Aug 4 21:03:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:42.608883 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7590d05-7b25-4ed3-9a9b-2a5dfba2a625 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:42.629438 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:42.647749 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e26e903-ba36-4e70-aa5f-ca4c151b48ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:42.648396 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e26e903-ba36-4e70-aa5f-ca4c151b48ce None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:42.648678 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e26e903-ba36-4e70-aa5f-ca4c151b48ce None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:42.648951 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e26e903-ba36-4e70-aa5f-ca4c151b48ce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:42.660238 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e26e903-ba36-4e70-aa5f-ca4c151b48ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:03:42.665671 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-4e26e903-ba36-4e70-aa5f-ca4c151b48ce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:03:42.665671 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:42.665671 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:42.665671 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:42.665671 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:42.665671 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:42.665671 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:42.665671 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:42.665671 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:42.665671 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:42.665671 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:42.665671 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:42.665671 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:03:42.665671 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:03:42.665671 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:03:42.665671 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:03:42.666788 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:03:42.666788 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:03:42.666788 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:03:42.666788 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:03:42.666788 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:42.666788 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:42.666788 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:42.666788 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:42.666788 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:42.666788 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:42.666788 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:42.666788 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:42.666788 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:42.666788 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:42.666788 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:42.666788 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:42.672539 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:42.672539 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:42.672539 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:42.672539 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:42.672539 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:03:42.672539 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:03:42.672539 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:03:42.672539 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:03:42.672539 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:03:42.672539 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:03:42.672539 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:03:42.672539 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:42.672539 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 420/831] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Aug 4 21:03:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:42.672539 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e26e903-ba36-4e70-aa5f-ca4c151b48ce None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:42.732705 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7590d05-7b25-4ed3-9a9b-2a5dfba2a625 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:42.774789 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62fb15e-43c9-4c5b-8262-a8247b78568b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:42.775441 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62fb15e-43c9-4c5b-8262-a8247b78568b None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:42.775725 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62fb15e-43c9-4c5b-8262-a8247b78568b None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:42.775885 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e26e903-ba36-4e70-aa5f-ca4c151b48ce None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:42.775991 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62fb15e-43c9-4c5b-8262-a8247b78568b None admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:42.776732 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62fb15e-43c9-4c5b-8262-a8247b78568b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:42.777038 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62fb15e-43c9-4c5b-8262-a8247b78568b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:42.779131 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62fb15e-43c9-4c5b-8262-a8247b78568b None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:42.797162 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 412/832] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Aug 4 21:03:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:42.798038 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62fb15e-43c9-4c5b-8262-a8247b78568b None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:42.814046 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88173020-0f8b-40e6-b748-b9f5ded42442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:42.814669 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88173020-0f8b-40e6-b748-b9f5ded42442 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:42.815022 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88173020-0f8b-40e6-b748-b9f5ded42442 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:42.815403 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88173020-0f8b-40e6-b748-b9f5ded42442 None admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:42.816155 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88173020-0f8b-40e6-b748-b9f5ded42442 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:42.816469 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88173020-0f8b-40e6-b748-b9f5ded42442 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:42.817939 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88173020-0f8b-40e6-b748-b9f5ded42442 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:42.831261 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 421/833] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Aug 4 21:03:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:42.832077 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88173020-0f8b-40e6-b748-b9f5ded42442 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:42.849354 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:42.875679 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:42.906495 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62fb15e-43c9-4c5b-8262-a8247b78568b None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:42.937020 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88173020-0f8b-40e6-b748-b9f5ded42442 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:42.942795 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927f55e8-9a2a-4f3e-889b-5d2107b2f759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:42.943448 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927f55e8-9a2a-4f3e-889b-5d2107b2f759 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:42.943735 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927f55e8-9a2a-4f3e-889b-5d2107b2f759 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:42.944007 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927f55e8-9a2a-4f3e-889b-5d2107b2f759 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:42.945043 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927f55e8-9a2a-4f3e-889b-5d2107b2f759 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:42.945043 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927f55e8-9a2a-4f3e-889b-5d2107b2f759 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:42.946374 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927f55e8-9a2a-4f3e-889b-5d2107b2f759 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:42.959119 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 413/834] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Aug 4 21:03:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:42.959119 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927f55e8-9a2a-4f3e-889b-5d2107b2f759 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:42.979053 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768f35ad-3724-4511-b6dc-735f365d7075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:42.979894 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768f35ad-3724-4511-b6dc-735f365d7075 None admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:42.980173 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768f35ad-3724-4511-b6dc-735f365d7075 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:42.980496 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768f35ad-3724-4511-b6dc-735f365d7075 None admin] PATH_INFO: `/v3/projects/17dc3d39e2c048ca9070588f5717a578/users/23ea380385f649628cc27e3603bc700c/roles/4eb30338d61e47808b51e050836643ea` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:42.998146 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768f35ad-3724-4511-b6dc-735f365d7075 None admin] RBAC: Authorizing `identity:create_grant(project_id=17dc3d39e2c048ca9070588f5717a578, user_id=23ea380385f649628cc27e3603bc700c, role_id=4eb30338d61e47808b51e050836643ea)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:42.999704 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768f35ad-3724-4511-b6dc-735f365d7075 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:43.059155 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927f55e8-9a2a-4f3e-889b-5d2107b2f759 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:43.060739 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:43.067942 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 422/835] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Aug 4 21:03:42 2022] PUT /identity/v3/projects/17dc3d39e2c048ca9070588f5717a578/users/23ea380385f649628cc27e3603bc700c/roles/4eb30338d61e47808b51e050836643ea => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:03:43.068768 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768f35ad-3724-4511-b6dc-735f365d7075 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:43.096481 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2713c89-e919-4d1d-8faa-0ea727ea648f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:43.097450 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2713c89-e919-4d1d-8faa-0ea727ea648f None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:43.098099 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2713c89-e919-4d1d-8faa-0ea727ea648f None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:43.098584 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2713c89-e919-4d1d-8faa-0ea727ea648f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:43.099654 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2713c89-e919-4d1d-8faa-0ea727ea648f None admin] RBAC: Adding query filter params (role.id=a432dd4735944ecfab2fa5ce1705d356, scope.project.id=17dc3d39e2c048ca9070588f5717a578, user.id=6a2ce1df87fa47e5be301916bf3245fb) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:43.100211 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2713c89-e919-4d1d-8faa-0ea727ea648f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:43.101847 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2713c89-e919-4d1d-8faa-0ea727ea648f None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:43.109584 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 414/836] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Aug 4 21:03:42 2022] GET /identity/v3/role_assignments?user.id=6a2ce1df87fa47e5be301916bf3245fb&scope.project.id=17dc3d39e2c048ca9070588f5717a578&role.id=a432dd4735944ecfab2fa5ce1705d356 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:43.111453 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9808d71-e631-4d99-bd01-20cd44444d92 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:43.111823 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9808d71-e631-4d99-bd01-20cd44444d92 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:43.112104 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9808d71-e631-4d99-bd01-20cd44444d92 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:43.113684 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 415/837] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:03:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:03:43.119474 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26aea17-7366-48fa-bdeb-87760dcab9b5 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:43.119885 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26aea17-7366-48fa-bdeb-87760dcab9b5 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:43.120635 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26aea17-7366-48fa-bdeb-87760dcab9b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:43.149188 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-d26aea17-7366-48fa-bdeb-87760dcab9b5 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:03:43.150768 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-d26aea17-7366-48fa-bdeb-87760dcab9b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:03:43.218889 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:43.239961 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768f35ad-3724-4511-b6dc-735f365d7075 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:43.249613 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 416/838] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:03:43 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 21:03:43.273788 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26aea17-7366-48fa-bdeb-87760dcab9b5 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:43.290234 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd521c71-235e-4cc8-a2fe-77bf3e4b98e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:43.291205 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd521c71-235e-4cc8-a2fe-77bf3e4b98e7 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:43.291825 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd521c71-235e-4cc8-a2fe-77bf3e4b98e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:43.292382 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd521c71-235e-4cc8-a2fe-77bf3e4b98e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:43.304556 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd521c71-235e-4cc8-a2fe-77bf3e4b98e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:43.306412 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd521c71-235e-4cc8-a2fe-77bf3e4b98e7 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:43.324065 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-dd521c71-235e-4cc8-a2fe-77bf3e4b98e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:43.324065 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:43.324065 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:43.324065 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:43.324065 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:43.324065 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:43.324065 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:43.324065 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:43.324065 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:43.324065 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:43.324065 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:43.324065 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:43.324065 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:03:43.324065 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:03:43.324065 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:03:43.325004 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:03:43.325004 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:43.325004 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:43.325004 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:43.325004 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:43.325004 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:43.325004 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:43.325004 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:43.325004 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:43.325004 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:43.325004 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:43.325004 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:43.325004 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:43.325004 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:43.325004 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:43.325004 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:43.325004 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:43.325884 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:03:43.325884 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:03:43.325884 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:43.325884 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:43.328003 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 423/839] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:03:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:43.337785 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd521c71-235e-4cc8-a2fe-77bf3e4b98e7 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:43.387879 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:43.398421 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26aea17-7366-48fa-bdeb-87760dcab9b5 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:43.450631 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a79c8e-4049-48e2-88e9-ebb0cf283e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:43.451429 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a79c8e-4049-48e2-88e9-ebb0cf283e3f None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:43.451798 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a79c8e-4049-48e2-88e9-ebb0cf283e3f None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:43.452138 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a79c8e-4049-48e2-88e9-ebb0cf283e3f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:43.453384 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a79c8e-4049-48e2-88e9-ebb0cf283e3f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:43.455800 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a79c8e-4049-48e2-88e9-ebb0cf283e3f None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:43.469841 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-01a79c8e-4049-48e2-88e9-ebb0cf283e3f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 21:03:43.469841 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:43.469841 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 21:03:43.469841 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 21:03:43.469841 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 21:03:43.469841 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 21:03:43.469841 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 21:03:43.469841 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:03:43.469841 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 21:03:43.469841 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:43.469841 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 21:03:43.469841 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:43.469841 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:43.469841 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:43.469841 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:43.469841 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:43.469841 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:43.471710 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:43.471710 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:43.471710 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:43.471710 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:43.471710 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:43.471710 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:43.471710 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 21:03:43.471710 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 21:03:43.471710 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 21:03:43.471710 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 21:03:43.471710 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:43.471710 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:43.471710 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:43.471710 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:43.471710 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:43.471710 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:43.476123 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:43.476123 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:43.476123 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:43.476123 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:43.476123 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:43.476123 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:43.476123 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:43.476123 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:43.476123 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:43.476123 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:43.476123 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 21:03:43.476123 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:03:43.476123 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 21:03:43.476123 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:43.476123 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 417/840] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Aug 4 21:03:43 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:43.480254 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a79c8e-4049-48e2-88e9-ebb0cf283e3f None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:43.486865 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd521c71-235e-4cc8-a2fe-77bf3e4b98e7 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:43.533358 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7751a4c4-c580-4add-9782-da851561df1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:43.534247 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7751a4c4-c580-4add-9782-da851561df1c None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:43.534528 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7751a4c4-c580-4add-9782-da851561df1c None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:43.534797 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7751a4c4-c580-4add-9782-da851561df1c None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:43.535841 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7751a4c4-c580-4add-9782-da851561df1c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:43.536204 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7751a4c4-c580-4add-9782-da851561df1c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:43.538254 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7751a4c4-c580-4add-9782-da851561df1c None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:43.550515 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:43.555912 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 424/841] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:03:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:03:43.561831 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7751a4c4-c580-4add-9782-da851561df1c None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:43.599559 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a79c8e-4049-48e2-88e9-ebb0cf283e3f None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:43.635953 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b87c06-526b-4113-97e4-da1237df68f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:43.636749 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b87c06-526b-4113-97e4-da1237df68f6 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:43.637074 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b87c06-526b-4113-97e4-da1237df68f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:43.637300 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b87c06-526b-4113-97e4-da1237df68f6 None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:43.637971 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b87c06-526b-4113-97e4-da1237df68f6 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:43.638272 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b87c06-526b-4113-97e4-da1237df68f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:43.639772 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b87c06-526b-4113-97e4-da1237df68f6 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:43.649858 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 418/842] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Aug 4 21:03:43 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:03:43.652645 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b87c06-526b-4113-97e4-da1237df68f6 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:43.684046 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7751a4c4-c580-4add-9782-da851561df1c None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:43.685248 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:43.731489 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e2e54f-2480-4e34-bf93-734a4af6985d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:43.732062 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e2e54f-2480-4e34-bf93-734a4af6985d None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:43.732340 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e2e54f-2480-4e34-bf93-734a4af6985d None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:43.732610 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e2e54f-2480-4e34-bf93-734a4af6985d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:43.741972 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e2e54f-2480-4e34-bf93-734a4af6985d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:43.743754 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e2e54f-2480-4e34-bf93-734a4af6985d None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:43.764478 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-80e2e54f-2480-4e34-bf93-734a4af6985d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 04 21:03:43.764478 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:43.764478 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:43.764478 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:43.764478 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:43.764478 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:43.764478 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:43.764478 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:43.764478 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:43.764478 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:43.764478 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:43.764478 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:43.764478 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 21:03:43.764478 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 21:03:43.764478 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 21:03:43.764478 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 21:03:43.765501 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:43.765501 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:43.765501 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 21:03:43.765501 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:03:43.765501 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 21:03:43.765501 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:03:43.765501 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:43.765501 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:43.765501 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:43.765501 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:43.765501 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:43.765501 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:43.765501 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:43.765501 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:43.765501 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:43.765501 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:43.765501 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:43.765501 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:43.767117 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:43.767117 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:43.767117 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 21:03:43.767117 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 21:03:43.767117 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 21:03:43.767117 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 21:03:43.767117 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 21:03:43.767117 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 21:03:43.767117 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 04 21:03:43.767117 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:43.767117 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 425/843] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Aug 4 21:03:43 2022] GET /identity/v3/users/swift => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:43.771147 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e2e54f-2480-4e34-bf93-734a4af6985d None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:43.779085 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b87c06-526b-4113-97e4-da1237df68f6 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:43.816394 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b91266-fb51-473d-a8f8-ca25e6001ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:43.817024 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b91266-fb51-473d-a8f8-ca25e6001ce8 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:43.817308 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b91266-fb51-473d-a8f8-ca25e6001ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:43.817582 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b91266-fb51-473d-a8f8-ca25e6001ce8 None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:43.818258 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b91266-fb51-473d-a8f8-ca25e6001ce8 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:43.818562 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b91266-fb51-473d-a8f8-ca25e6001ce8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:43.820126 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b91266-fb51-473d-a8f8-ca25e6001ce8 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:43.831347 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 419/844] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Aug 4 21:03:43 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:43.850335 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:43.898857 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e2e54f-2480-4e34-bf93-734a4af6985d None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:43.937505 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093b07e1-cd7c-4360-b661-2ef9990b58f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:43.938149 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093b07e1-cd7c-4360-b661-2ef9990b58f9 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:43.938473 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093b07e1-cd7c-4360-b661-2ef9990b58f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:43.938749 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093b07e1-cd7c-4360-b661-2ef9990b58f9 None admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:43.939716 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093b07e1-cd7c-4360-b661-2ef9990b58f9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:43.940023 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093b07e1-cd7c-4360-b661-2ef9990b58f9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:43.941487 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093b07e1-cd7c-4360-b661-2ef9990b58f9 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:43.962744 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 426/845] 10.222.0.25 () {56 vars in 1135 bytes} [Thu Aug 4 21:03:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:03:43.968605 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b91266-fb51-473d-a8f8-ca25e6001ce8 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:44.061573 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:44.074227 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b91266-fb51-473d-a8f8-ca25e6001ce8 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:44.109763 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c14ac20-b603-4cca-96df-ed1ecea84726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:44.110391 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c14ac20-b603-4cca-96df-ed1ecea84726 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:44.110673 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c14ac20-b603-4cca-96df-ed1ecea84726 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:44.110976 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c14ac20-b603-4cca-96df-ed1ecea84726 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:44.180395 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c14ac20-b603-4cca-96df-ed1ecea84726 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:44.181970 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c14ac20-b603-4cca-96df-ed1ecea84726 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:44.275732 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:44.288776 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 420/846] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Aug 4 21:03:43 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 21:03:44.294460 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093b07e1-cd7c-4360-b661-2ef9990b58f9 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:44.409664 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093b07e1-cd7c-4360-b661-2ef9990b58f9 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:44.427729 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:44.445257 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81de450b-4cde-4bad-92cf-8874b3f028eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:44.445936 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81de450b-4cde-4bad-92cf-8874b3f028eb None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:44.446218 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81de450b-4cde-4bad-92cf-8874b3f028eb None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:44.462848 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81de450b-4cde-4bad-92cf-8874b3f028eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:44.462848 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81de450b-4cde-4bad-92cf-8874b3f028eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:44.462848 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81de450b-4cde-4bad-92cf-8874b3f028eb None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:44.473365 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-81de450b-4cde-4bad-92cf-8874b3f028eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:44.473365 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:44.473365 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:44.473365 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:44.473365 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:44.473365 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:44.473365 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:44.473365 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:44.473365 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:44.473365 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:44.473365 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:44.473365 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:44.473365 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:03:44.473365 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:03:44.473365 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:03:44.474372 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:03:44.474372 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:44.474372 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:44.474372 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:44.474372 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:44.474372 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:44.474372 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:44.474372 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:44.474372 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:44.474372 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:44.474372 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:44.474372 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:44.474372 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:44.474372 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:44.474372 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:44.474372 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:44.474372 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:44.475276 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:03:44.475276 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:03:44.475276 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:44.475276 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:44.475276 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 427/847] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:03:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:44.480446 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c14ac20-b603-4cca-96df-ed1ecea84726 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:44.581688 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c14ac20-b603-4cca-96df-ed1ecea84726 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:44.616610 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1777e1d1-7524-4cc6-ad21-c7a6c6404777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:44.617362 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1777e1d1-7524-4cc6-ad21-c7a6c6404777 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:44.617581 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1777e1d1-7524-4cc6-ad21-c7a6c6404777 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:44.617858 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1777e1d1-7524-4cc6-ad21-c7a6c6404777 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:44.618557 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1777e1d1-7524-4cc6-ad21-c7a6c6404777 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:44.618862 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1777e1d1-7524-4cc6-ad21-c7a6c6404777 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:44.620329 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1777e1d1-7524-4cc6-ad21-c7a6c6404777 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:44.638064 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 421/848] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:03:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:44.643492 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81de450b-4cde-4bad-92cf-8874b3f028eb None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:44.757182 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81de450b-4cde-4bad-92cf-8874b3f028eb None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:44.793097 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee441f1-ad40-4c1a-a32c-89511991dd52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:44.793745 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee441f1-ad40-4c1a-a32c-89511991dd52 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:44.794081 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee441f1-ad40-4c1a-a32c-89511991dd52 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:44.794360 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee441f1-ad40-4c1a-a32c-89511991dd52 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:44.801332 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee441f1-ad40-4c1a-a32c-89511991dd52 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:44.802788 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee441f1-ad40-4c1a-a32c-89511991dd52 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:44.814615 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-bee441f1-ad40-4c1a-a32c-89511991dd52 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:03:44.814615 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:44.814615 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:44.814615 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:44.814615 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:44.814615 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:44.814615 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:44.814615 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:44.814615 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:44.814615 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:44.814615 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:44.814615 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:44.814615 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 21:03:44.814615 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 21:03:44.814615 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 21:03:44.815557 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 21:03:44.815557 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:44.815557 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:44.815557 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:44.815557 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:44.815557 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:44.815557 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:44.815557 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:44.815557 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:44.815557 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:44.815557 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:44.815557 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:44.815557 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:44.815557 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:44.815557 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:44.815557 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:44.815557 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:44.816454 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 21:03:44.816454 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 21:03:44.816454 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 21:03:44.816454 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 21:03:44.816454 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 21:03:44.816454 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 21:03:44.816454 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:03:44.816454 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:44.816454 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 428/849] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Aug 4 21:03:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:44.821982 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1777e1d1-7524-4cc6-ad21-c7a6c6404777 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:44.879695 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:44.926545 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1777e1d1-7524-4cc6-ad21-c7a6c6404777 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:44.961150 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa033f4d-2152-4ae8-813e-1dcf6d87930a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:44.961778 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa033f4d-2152-4ae8-813e-1dcf6d87930a None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:44.962063 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa033f4d-2152-4ae8-813e-1dcf6d87930a None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:44.962340 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa033f4d-2152-4ae8-813e-1dcf6d87930a None admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:44.963146 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa033f4d-2152-4ae8-813e-1dcf6d87930a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:44.963554 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa033f4d-2152-4ae8-813e-1dcf6d87930a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:44.965016 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa033f4d-2152-4ae8-813e-1dcf6d87930a None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:44.978734 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 422/850] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Aug 4 21:03:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:44.984859 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee441f1-ad40-4c1a-a32c-89511991dd52 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:45.023786 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:45.102780 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee441f1-ad40-4c1a-a32c-89511991dd52 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:45.137868 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50121220-85ab-4a3a-84f6-33256f7979fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:45.138496 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50121220-85ab-4a3a-84f6-33256f7979fd None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:45.138785 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50121220-85ab-4a3a-84f6-33256f7979fd None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:45.139066 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50121220-85ab-4a3a-84f6-33256f7979fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:45.139966 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50121220-85ab-4a3a-84f6-33256f7979fd None admin] RBAC: Adding query filter params (role.id=a432dd4735944ecfab2fa5ce1705d356, scope.project.id=17dc3d39e2c048ca9070588f5717a578, user.id=58935367c0ad445f8477cdac0df04d0a) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:45.140691 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50121220-85ab-4a3a-84f6-33256f7979fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:45.141113 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666853bb-46e9-44d2-b61c-116af1074cc6 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:45.141405 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666853bb-46e9-44d2-b61c-116af1074cc6 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:45.141683 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666853bb-46e9-44d2-b61c-116af1074cc6 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:45.142585 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50121220-85ab-4a3a-84f6-33256f7979fd None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:45.143334 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 423/851] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:03:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 21:03:45.148034 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eabf7e7-613b-454f-a327-f1256c875ec2 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:45.148314 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eabf7e7-613b-454f-a327-f1256c875ec2 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:45.148583 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eabf7e7-613b-454f-a327-f1256c875ec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:45.149864 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 429/852] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Aug 4 21:03:44 2022] GET /identity/v3/role_assignments?user.id=58935367c0ad445f8477cdac0df04d0a&scope.project.id=17dc3d39e2c048ca9070588f5717a578&role.id=a432dd4735944ecfab2fa5ce1705d356 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:45.168430 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-8eabf7e7-613b-454f-a327-f1256c875ec2 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:03:45.169757 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-8eabf7e7-613b-454f-a327-f1256c875ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:03:45.245073 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 424/853] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:03:45 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 21:03:45.268995 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50121220-85ab-4a3a-84f6-33256f7979fd None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:45.346364 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460b3c09-fd58-4fe4-886b-09abe8aa8419 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:45.346694 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460b3c09-fd58-4fe4-886b-09abe8aa8419 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:45.346845 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460b3c09-fd58-4fe4-886b-09abe8aa8419 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:45.348423 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 425/854] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:03:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:03:45.361927 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c959d6e5-0562-413a-a43f-fe3c585feca6 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:45.361927 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c959d6e5-0562-413a-a43f-fe3c585feca6 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:45.361927 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c959d6e5-0562-413a-a43f-fe3c585feca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:45.382293 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-c959d6e5-0562-413a-a43f-fe3c585feca6 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:03:45.384039 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:45.385593 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-c959d6e5-0562-413a-a43f-fe3c585feca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:03:45.391635 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50121220-85ab-4a3a-84f6-33256f7979fd None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:45.436399 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:45.444963 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baf993c-593e-40f1-b804-e39297a51265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:45.449192 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baf993c-593e-40f1-b804-e39297a51265 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:45.449192 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baf993c-593e-40f1-b804-e39297a51265 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:45.449192 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baf993c-593e-40f1-b804-e39297a51265 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:45.461082 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baf993c-593e-40f1-b804-e39297a51265 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:03:45.467690 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-0baf993c-593e-40f1-b804-e39297a51265 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:03:45.467690 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:45.467690 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:45.467690 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:45.467690 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:45.467690 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:45.467690 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:45.467690 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:45.467690 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:45.467690 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:45.467690 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:45.467690 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:45.467690 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:03:45.467690 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:03:45.467690 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:03:45.467690 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:03:45.468663 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:03:45.468663 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:03:45.468663 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:03:45.468663 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:03:45.468663 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:45.468663 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:45.468663 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:45.468663 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:45.468663 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:45.468663 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:45.468663 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:45.468663 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:45.468663 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:45.468663 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:45.468663 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:45.468663 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:45.469503 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:45.469503 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:45.469503 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:45.469503 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:45.469503 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:03:45.469503 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:03:45.469503 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:03:45.469503 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:03:45.469503 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:03:45.469503 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:03:45.469503 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:03:45.469503 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:45.473063 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 430/855] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Aug 4 21:03:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:45.479370 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baf993c-593e-40f1-b804-e39297a51265 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:45.486315 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 426/856] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:03:45 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:03:45.508055 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c959d6e5-0562-413a-a43f-fe3c585feca6 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:45.591522 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baf993c-593e-40f1-b804-e39297a51265 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:45.630949 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c959d6e5-0562-413a-a43f-fe3c585feca6 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:45.631641 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d830c9-6d66-49d9-a211-562c0404f4c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:45.632268 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d830c9-6d66-49d9-a211-562c0404f4c8 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:45.632685 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d830c9-6d66-49d9-a211-562c0404f4c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:45.632957 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d830c9-6d66-49d9-a211-562c0404f4c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:45.633785 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d830c9-6d66-49d9-a211-562c0404f4c8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:45.634088 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d830c9-6d66-49d9-a211-562c0404f4c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:45.635542 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d830c9-6d66-49d9-a211-562c0404f4c8 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:45.649818 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 431/857] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Aug 4 21:03:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:03:45.650943 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d830c9-6d66-49d9-a211-562c0404f4c8 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:45.670911 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:45.671524 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01015fb4-76bc-4f84-82b4-f0c37a3c9d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:45.673946 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01015fb4-76bc-4f84-82b4-f0c37a3c9d7b None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:45.674420 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01015fb4-76bc-4f84-82b4-f0c37a3c9d7b None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:45.674896 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01015fb4-76bc-4f84-82b4-f0c37a3c9d7b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:45.676357 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01015fb4-76bc-4f84-82b4-f0c37a3c9d7b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:45.678546 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01015fb4-76bc-4f84-82b4-f0c37a3c9d7b None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:45.691915 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-01015fb4-76bc-4f84-82b4-f0c37a3c9d7b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 21:03:45.691915 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:45.691915 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 21:03:45.691915 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 21:03:45.691915 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 21:03:45.691915 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 21:03:45.691915 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 21:03:45.691915 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:03:45.691915 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 21:03:45.691915 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:45.691915 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 21:03:45.691915 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:45.691915 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:45.691915 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:45.691915 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:45.691915 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:45.699851 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:45.699851 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:45.699851 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:45.699851 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:45.699851 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:45.699851 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:45.699851 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:45.699851 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 21:03:45.699851 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 21:03:45.699851 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 21:03:45.699851 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 21:03:45.699851 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:45.699851 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:45.699851 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:45.699851 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:45.699851 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:45.700699 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:45.700699 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:45.700699 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:45.700699 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:45.700699 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:45.700699 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:45.700699 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:45.700699 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:45.700699 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:45.700699 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:45.700699 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:45.700699 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 21:03:45.700699 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:03:45.700699 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 21:03:45.700699 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:45.700699 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 427/858] 10.222.0.25 () {56 vars in 1132 bytes} [Thu Aug 4 21:03:45 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:45.702045 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01015fb4-76bc-4f84-82b4-f0c37a3c9d7b None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:45.740095 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:45.792246 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d830c9-6d66-49d9-a211-562c0404f4c8 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:45.813122 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01015fb4-76bc-4f84-82b4-f0c37a3c9d7b None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:45.832651 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265ba9e7-5f61-43bc-8c1f-7c963d45c267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:45.833671 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265ba9e7-5f61-43bc-8c1f-7c963d45c267 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:45.834162 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265ba9e7-5f61-43bc-8c1f-7c963d45c267 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:45.834661 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265ba9e7-5f61-43bc-8c1f-7c963d45c267 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:45.846614 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265ba9e7-5f61-43bc-8c1f-7c963d45c267 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:45.848704 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265ba9e7-5f61-43bc-8c1f-7c963d45c267 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:45.849618 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd317302-1375-4441-a3dd-b4a409ef1278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:45.850243 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd317302-1375-4441-a3dd-b4a409ef1278 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:45.850526 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd317302-1375-4441-a3dd-b4a409ef1278 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:45.850798 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd317302-1375-4441-a3dd-b4a409ef1278 None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:45.851502 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd317302-1375-4441-a3dd-b4a409ef1278 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:45.851920 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd317302-1375-4441-a3dd-b4a409ef1278 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:45.853177 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd317302-1375-4441-a3dd-b4a409ef1278 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:45.862914 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 428/859] 10.222.0.25 () {56 vars in 1126 bytes} [Thu Aug 4 21:03:45 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:03:45.864139 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd317302-1375-4441-a3dd-b4a409ef1278 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:45.874354 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-265ba9e7-5f61-43bc-8c1f-7c963d45c267 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:45.874354 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:45.874354 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:45.874354 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:45.874354 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:45.874354 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:45.874354 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:45.874354 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:45.874354 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:45.874354 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:45.874354 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:45.874354 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:45.874354 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:03:45.874354 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:03:45.874354 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:03:45.875253 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:03:45.875253 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:45.875253 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:45.875253 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:45.875253 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:45.875253 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:45.875253 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:45.875253 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:45.875253 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:45.875253 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:45.875253 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:45.875253 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:45.875253 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:45.875253 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:45.875253 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:45.875253 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:45.875253 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:45.876188 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:03:45.876188 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:03:45.876188 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:45.876188 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:45.878915 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 432/860] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:03:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:45.881250 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265ba9e7-5f61-43bc-8c1f-7c963d45c267 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:45.969020 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:45.988891 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:45.995735 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd317302-1375-4441-a3dd-b4a409ef1278 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:46.009951 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265ba9e7-5f61-43bc-8c1f-7c963d45c267 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:46.031294 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713c8d2b-1771-4851-9817-cc580f0b9874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:46.031950 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713c8d2b-1771-4851-9817-cc580f0b9874 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:46.032237 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713c8d2b-1771-4851-9817-cc580f0b9874 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:46.032515 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713c8d2b-1771-4851-9817-cc580f0b9874 None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:46.033879 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713c8d2b-1771-4851-9817-cc580f0b9874 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:46.034447 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713c8d2b-1771-4851-9817-cc580f0b9874 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:46.035892 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713c8d2b-1771-4851-9817-cc580f0b9874 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:46.040673 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 429/861] 10.222.0.25 () {56 vars in 1126 bytes} [Thu Aug 4 21:03:45 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:03:46.045130 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83741874-b483-4109-b3ec-29b846eb9556 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:46.045792 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83741874-b483-4109-b3ec-29b846eb9556 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:46.046130 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d7012a-ce3e-493b-928d-449b7b3096f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:46.046540 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83741874-b483-4109-b3ec-29b846eb9556 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:46.046775 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d7012a-ce3e-493b-928d-449b7b3096f4 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:46.047066 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d7012a-ce3e-493b-928d-449b7b3096f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:46.047368 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d7012a-ce3e-493b-928d-449b7b3096f4 None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:46.048111 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d7012a-ce3e-493b-928d-449b7b3096f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:46.048995 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d7012a-ce3e-493b-928d-449b7b3096f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:46.050444 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 430/862] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:03:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:03:46.054834 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d7012a-ce3e-493b-928d-449b7b3096f4 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:46.056387 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92caa0c-8272-4d42-a9f3-1a427f1a7e86 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:46.057236 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92caa0c-8272-4d42-a9f3-1a427f1a7e86 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:46.057718 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92caa0c-8272-4d42-a9f3-1a427f1a7e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:46.077826 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 433/863] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:03:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:46.079459 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d7012a-ce3e-493b-928d-449b7b3096f4 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:46.082983 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-d92caa0c-8272-4d42-a9f3-1a427f1a7e86 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:03:46.084950 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-d92caa0c-8272-4d42-a9f3-1a427f1a7e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:03:46.167763 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 431/864] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:03:46 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 21:03:46.185217 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92caa0c-8272-4d42-a9f3-1a427f1a7e86 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:46.236861 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d7012a-ce3e-493b-928d-449b7b3096f4 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:46.265978 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:46.275152 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e287d8-55a0-488f-a401-b0eeccce6978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:46.276075 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e287d8-55a0-488f-a401-b0eeccce6978 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:46.276529 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e287d8-55a0-488f-a401-b0eeccce6978 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:46.276975 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e287d8-55a0-488f-a401-b0eeccce6978 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:46.285432 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e287d8-55a0-488f-a401-b0eeccce6978 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:46.287399 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e287d8-55a0-488f-a401-b0eeccce6978 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:46.300364 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-75e287d8-55a0-488f-a401-b0eeccce6978 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 21:03:46.300364 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:46.300364 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:46.300364 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:46.300364 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:46.300364 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:46.300364 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:46.300364 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:46.300364 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:46.300364 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:46.300364 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:46.300364 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:46.300364 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 21:03:46.300364 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 21:03:46.300364 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 21:03:46.300364 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 21:03:46.301688 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:46.301688 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:46.301688 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 21:03:46.301688 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:03:46.301688 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 21:03:46.301688 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:03:46.301688 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:46.301688 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:46.301688 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:46.301688 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:46.301688 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:46.301688 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:46.301688 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:46.301688 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:46.301688 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:46.301688 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:46.301688 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:46.301688 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:46.302636 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:46.302636 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:46.302636 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 21:03:46.302636 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 21:03:46.302636 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 21:03:46.302636 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 21:03:46.302636 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 21:03:46.302636 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 21:03:46.302636 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 21:03:46.302636 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:46.305546 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 434/865] 10.222.0.25 () {56 vars in 1117 bytes} [Thu Aug 4 21:03:46 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:46.312769 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e287d8-55a0-488f-a401-b0eeccce6978 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:46.315710 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:46.330883 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92caa0c-8272-4d42-a9f3-1a427f1a7e86 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:46.374793 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f0a1ec-8edd-4ddd-8c77-bb23a3cc7b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:46.375455 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f0a1ec-8edd-4ddd-8c77-bb23a3cc7b62 None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:46.375749 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f0a1ec-8edd-4ddd-8c77-bb23a3cc7b62 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:46.376017 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f0a1ec-8edd-4ddd-8c77-bb23a3cc7b62 None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:46.376800 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f0a1ec-8edd-4ddd-8c77-bb23a3cc7b62 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:46.378182 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f0a1ec-8edd-4ddd-8c77-bb23a3cc7b62 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:46.432340 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e287d8-55a0-488f-a401-b0eeccce6978 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:46.449887 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 432/866] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Aug 4 21:03:46 2022] POST /identity/v3/services => generated 251 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:03:46.482697 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301f5d60-aa90-4335-8b8e-92c9f452b7a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:46.483368 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f5d60-aa90-4335-8b8e-92c9f452b7a8 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:46.483659 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f5d60-aa90-4335-8b8e-92c9f452b7a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:46.483975 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f5d60-aa90-4335-8b8e-92c9f452b7a8 None admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:46.484864 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301f5d60-aa90-4335-8b8e-92c9f452b7a8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:46.485181 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301f5d60-aa90-4335-8b8e-92c9f452b7a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:46.487219 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301f5d60-aa90-4335-8b8e-92c9f452b7a8 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:46.509534 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 435/867] 10.222.0.25 () {56 vars in 1149 bytes} [Thu Aug 4 21:03:46 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:46.515342 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f0a1ec-8edd-4ddd-8c77-bb23a3cc7b62 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:46.632980 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f0a1ec-8edd-4ddd-8c77-bb23a3cc7b62 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:46.668965 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca98f12-2235-46d5-9590-ca0fc7327522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:46.669739 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca98f12-2235-46d5-9590-ca0fc7327522 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:46.670095 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca98f12-2235-46d5-9590-ca0fc7327522 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:46.670440 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca98f12-2235-46d5-9590-ca0fc7327522 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:46.690936 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:46.742078 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca98f12-2235-46d5-9590-ca0fc7327522 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:46.743786 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca98f12-2235-46d5-9590-ca0fc7327522 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:46.847096 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 433/868] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Aug 4 21:03:46 2022] GET /identity/v3/auth/tokens => generated 1262 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 21:03:46.852686 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301f5d60-aa90-4335-8b8e-92c9f452b7a8 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:46.898610 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:46.958959 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301f5d60-aa90-4335-8b8e-92c9f452b7a8 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:46.997748 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3933545-36d5-4246-a32d-4a1a800fcac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:46.998407 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3933545-36d5-4246-a32d-4a1a800fcac3 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:46.998692 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3933545-36d5-4246-a32d-4a1a800fcac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:46.998969 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3933545-36d5-4246-a32d-4a1a800fcac3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:47.010551 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3933545-36d5-4246-a32d-4a1a800fcac3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:47.012172 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3933545-36d5-4246-a32d-4a1a800fcac3 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:47.027977 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-e3933545-36d5-4246-a32d-4a1a800fcac3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:47.027977 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:47.027977 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:47.027977 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:47.027977 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:47.027977 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:47.027977 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:47.027977 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:47.027977 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:47.027977 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:47.027977 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:47.027977 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:47.027977 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:03:47.027977 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:03:47.027977 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:03:47.028885 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:03:47.028885 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:47.028885 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:47.028885 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:47.028885 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:47.028885 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:47.028885 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:47.028885 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:47.028885 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:47.028885 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:47.028885 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:47.028885 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:47.028885 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:47.028885 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:47.028885 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:47.028885 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:47.028885 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:47.029785 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:03:47.029785 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:03:47.029785 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:47.029785 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:47.029785 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 436/869] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:03:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:47.036116 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca98f12-2235-46d5-9590-ca0fc7327522 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:47.086788 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:47.143245 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca98f12-2235-46d5-9590-ca0fc7327522 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:47.180263 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dced581f-df48-48ef-8fc9-352ce4e3a0f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:47.180897 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dced581f-df48-48ef-8fc9-352ce4e3a0f5 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:47.181183 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dced581f-df48-48ef-8fc9-352ce4e3a0f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:47.181456 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dced581f-df48-48ef-8fc9-352ce4e3a0f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:47.182154 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dced581f-df48-48ef-8fc9-352ce4e3a0f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:47.182461 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dced581f-df48-48ef-8fc9-352ce4e3a0f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:47.183925 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dced581f-df48-48ef-8fc9-352ce4e3a0f5 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:47.196641 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 434/870] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:03:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:47.201897 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3933545-36d5-4246-a32d-4a1a800fcac3 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:47.310076 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f4ed2c-8d5d-400a-a24b-1559da8f85d8 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:47.310407 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f4ed2c-8d5d-400a-a24b-1559da8f85d8 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:47.310640 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f4ed2c-8d5d-400a-a24b-1559da8f85d8 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:47.313140 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 435/871] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:03:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 21:03:47.318818 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc1355-246d-46de-be4e-eb84902c780c None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:47.319363 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc1355-246d-46de-be4e-eb84902c780c None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:47.319841 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc1355-246d-46de-be4e-eb84902c780c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:47.320290 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:47.327149 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3933545-36d5-4246-a32d-4a1a800fcac3 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:47.341977 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-45bc1355-246d-46de-be4e-eb84902c780c None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:03:47.344086 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-45bc1355-246d-46de-be4e-eb84902c780c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:03:47.368329 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db328002-2d42-44c6-86be-22e88942a254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:47.368993 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db328002-2d42-44c6-86be-22e88942a254 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:47.369529 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db328002-2d42-44c6-86be-22e88942a254 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:47.369813 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db328002-2d42-44c6-86be-22e88942a254 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:47.379261 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db328002-2d42-44c6-86be-22e88942a254 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:47.381039 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db328002-2d42-44c6-86be-22e88942a254 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:47.396230 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-db328002-2d42-44c6-86be-22e88942a254 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:03:47.396230 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:47.396230 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:47.396230 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:47.396230 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:47.396230 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:47.396230 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:47.396230 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:47.396230 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:47.396230 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:47.396230 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:47.396230 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:47.396230 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 21:03:47.396230 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 21:03:47.396230 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 21:03:47.397161 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 21:03:47.397161 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:47.397161 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:47.397161 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:47.397161 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:47.397161 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:47.397161 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:47.397161 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:47.397161 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:47.397161 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:47.397161 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:47.397161 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:47.397161 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:47.397161 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:47.397161 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:47.397161 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:47.397161 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:47.398074 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 21:03:47.398074 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 21:03:47.398074 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 21:03:47.398074 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 21:03:47.398074 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 21:03:47.398074 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 21:03:47.398074 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:03:47.398074 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:47.398074 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 437/872] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Aug 4 21:03:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:47.402873 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db328002-2d42-44c6-86be-22e88942a254 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:47.436427 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 436/873] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:03:47 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 21:03:47.462375 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bc1355-246d-46de-be4e-eb84902c780c None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:47.510271 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db328002-2d42-44c6-86be-22e88942a254 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:47.511785 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:47.545746 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf756d1-a0ab-4e97-b266-5a8c65c59742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:47.546427 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf756d1-a0ab-4e97-b266-5a8c65c59742 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:47.546760 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf756d1-a0ab-4e97-b266-5a8c65c59742 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:47.547053 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf756d1-a0ab-4e97-b266-5a8c65c59742 None admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:47.547813 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf756d1-a0ab-4e97-b266-5a8c65c59742 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:47.548114 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf756d1-a0ab-4e97-b266-5a8c65c59742 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:47.549548 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf756d1-a0ab-4e97-b266-5a8c65c59742 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:47.561452 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 438/874] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Aug 4 21:03:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:03:47.567574 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf756d1-a0ab-4e97-b266-5a8c65c59742 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:47.570402 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bc1355-246d-46de-be4e-eb84902c780c None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:47.584242 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:47.608334 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d172d6-232b-4e71-adef-46f00f791a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:47.609356 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d172d6-232b-4e71-adef-46f00f791a6f None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:47.609863 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d172d6-232b-4e71-adef-46f00f791a6f None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:47.610389 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d172d6-232b-4e71-adef-46f00f791a6f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:47.611833 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d172d6-232b-4e71-adef-46f00f791a6f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:47.613464 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d172d6-232b-4e71-adef-46f00f791a6f None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:47.623104 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-d6d172d6-232b-4e71-adef-46f00f791a6f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 21:03:47.623104 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:47.623104 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 21:03:47.623104 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 21:03:47.623104 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 21:03:47.623104 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 21:03:47.623104 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 21:03:47.623104 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:03:47.623104 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 21:03:47.623104 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:47.623104 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 21:03:47.623104 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:47.623104 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:47.623104 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:47.623104 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:47.623104 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:47.623104 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:47.624147 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:47.624147 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:47.624147 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:47.624147 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:47.624147 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:47.624147 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:47.624147 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 21:03:47.624147 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 21:03:47.624147 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 21:03:47.624147 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 21:03:47.624147 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:47.624147 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:47.624147 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:47.624147 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:47.624147 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:47.624147 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:47.624999 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:47.624999 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:47.624999 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:47.624999 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:47.624999 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:47.624999 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:47.624999 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:47.624999 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:47.624999 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:47.624999 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:47.624999 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 21:03:47.624999 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:03:47.624999 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 21:03:47.624999 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:47.632945 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 437/875] 10.222.0.25 () {56 vars in 1126 bytes} [Thu Aug 4 21:03:47 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:47.634064 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d172d6-232b-4e71-adef-46f00f791a6f None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:47.670818 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf756d1-a0ab-4e97-b266-5a8c65c59742 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:47.708552 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a244194-e794-45a3-bb47-adaecb0bee02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:47.709179 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a244194-e794-45a3-bb47-adaecb0bee02 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:47.709459 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a244194-e794-45a3-bb47-adaecb0bee02 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:47.709730 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a244194-e794-45a3-bb47-adaecb0bee02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:47.710528 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a244194-e794-45a3-bb47-adaecb0bee02 None admin] RBAC: Adding query filter params (role.id=4eb30338d61e47808b51e050836643ea, scope.project.id=17dc3d39e2c048ca9070588f5717a578, user.id=23ea380385f649628cc27e3603bc700c) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:47.710839 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a244194-e794-45a3-bb47-adaecb0bee02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:47.712445 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a244194-e794-45a3-bb47-adaecb0bee02 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:47.723880 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 439/876] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Aug 4 21:03:47 2022] GET /identity/v3/role_assignments?user.id=23ea380385f649628cc27e3603bc700c&scope.project.id=17dc3d39e2c048ca9070588f5717a578&role.id=4eb30338d61e47808b51e050836643ea => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:03:47.750700 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d172d6-232b-4e71-adef-46f00f791a6f None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:47.763819 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:47.790389 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cb49d6-8f97-4d03-8566-3f6de50065bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:47.791521 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb49d6-8f97-4d03-8566-3f6de50065bb None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:47.792011 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb49d6-8f97-4d03-8566-3f6de50065bb None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:47.792510 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb49d6-8f97-4d03-8566-3f6de50065bb None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:47.793456 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cb49d6-8f97-4d03-8566-3f6de50065bb None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:47.793968 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cb49d6-8f97-4d03-8566-3f6de50065bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:47.795689 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cb49d6-8f97-4d03-8566-3f6de50065bb None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:47.802868 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 438/877] 10.222.0.25 () {56 vars in 1122 bytes} [Thu Aug 4 21:03:47 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:47.812491 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a244194-e794-45a3-bb47-adaecb0bee02 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:47.897753 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:47.920300 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a244194-e794-45a3-bb47-adaecb0bee02 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:47.956980 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5e0f41-948f-46de-92ed-bd4d5c020bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:47.957640 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5e0f41-948f-46de-92ed-bd4d5c020bf8 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:47.957891 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5e0f41-948f-46de-92ed-bd4d5c020bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:47.958165 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5e0f41-948f-46de-92ed-bd4d5c020bf8 None admin] PATH_INFO: `/v3/services` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:47.958846 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5e0f41-948f-46de-92ed-bd4d5c020bf8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:47.959149 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5e0f41-948f-46de-92ed-bd4d5c020bf8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:47.960625 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5e0f41-948f-46de-92ed-bd4d5c020bf8 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:47.965646 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 440/878] 10.222.0.25 () {56 vars in 1122 bytes} [Thu Aug 4 21:03:47 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:48.200715 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c056b6-9ba3-4bd5-8e60-edd5130527c5 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:48.201004 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c056b6-9ba3-4bd5-8e60-edd5130527c5 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:48.201293 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c056b6-9ba3-4bd5-8e60-edd5130527c5 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:48.203010 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 439/879] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:03:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:03:48.209029 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a15b54a-c4bc-42d3-8edb-691b7abe78d0 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:48.209329 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a15b54a-c4bc-42d3-8edb-691b7abe78d0 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:48.209615 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a15b54a-c4bc-42d3-8edb-691b7abe78d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:48.237706 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-2a15b54a-c4bc-42d3-8edb-691b7abe78d0 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:03:48.238913 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-2a15b54a-c4bc-42d3-8edb-691b7abe78d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:03:48.310547 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 441/880] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:03:48 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:03:48.332621 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c056b6-9ba3-4bd5-8e60-edd5130527c5 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:48.442442 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c056b6-9ba3-4bd5-8e60-edd5130527c5 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:48.477265 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ac9684-fc64-4219-bb41-19aa078db368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:48.477811 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ac9684-fc64-4219-bb41-19aa078db368 None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:48.478092 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ac9684-fc64-4219-bb41-19aa078db368 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:48.478367 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ac9684-fc64-4219-bb41-19aa078db368 None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:48.479055 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ac9684-fc64-4219-bb41-19aa078db368 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:48.480467 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ac9684-fc64-4219-bb41-19aa078db368 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:48.605505 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 440/881] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Aug 4 21:03:48 2022] POST /identity/v3/services => generated 269 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:03:48.612127 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c028e61e-921a-4c26-9d99-85f6523b7cec None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:48.612650 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c028e61e-921a-4c26-9d99-85f6523b7cec None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:48.612726 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c028e61e-921a-4c26-9d99-85f6523b7cec None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:48.614388 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 442/882] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:03:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:03:48.620156 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a2054b-5425-4560-834b-cfbb9b7c2918 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:48.621300 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a2054b-5425-4560-834b-cfbb9b7c2918 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:48.621300 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a2054b-5425-4560-834b-cfbb9b7c2918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:48.640999 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-60a2054b-5425-4560-834b-cfbb9b7c2918 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:03:48.642819 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-60a2054b-5425-4560-834b-cfbb9b7c2918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:03:48.670335 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:48.725154 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 441/883] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:03:48 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:03:48.748042 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c028e61e-921a-4c26-9d99-85f6523b7cec None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:48.826865 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:48.857745 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c028e61e-921a-4c26-9d99-85f6523b7cec None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:48.893968 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59b8bc9-c281-4d8c-8589-92a84205f81a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:48.894614 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59b8bc9-c281-4d8c-8589-92a84205f81a None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:48.894896 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59b8bc9-c281-4d8c-8589-92a84205f81a None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:48.895165 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59b8bc9-c281-4d8c-8589-92a84205f81a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:48.895828 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59b8bc9-c281-4d8c-8589-92a84205f81a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:48.897201 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59b8bc9-c281-4d8c-8589-92a84205f81a None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:48.906255 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-a59b8bc9-c281-4d8c-8589-92a84205f81a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 21:03:48.906255 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:48.906255 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 21:03:48.906255 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 21:03:48.906255 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 21:03:48.906255 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 21:03:48.906255 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 21:03:48.906255 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:03:48.906255 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 21:03:48.906255 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:48.906255 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 21:03:48.906255 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:48.906255 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:48.906255 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:48.906255 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:48.906255 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:48.906255 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:48.907274 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:48.907274 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:48.907274 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:48.907274 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:48.907274 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:48.907274 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:48.907274 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 21:03:48.907274 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 21:03:48.907274 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 21:03:48.907274 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 21:03:48.907274 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:48.907274 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:48.907274 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:48.907274 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:48.907274 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:48.907274 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:48.908195 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:48.908195 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:48.908195 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:48.908195 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:48.908195 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:48.908195 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:48.908195 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:48.908195 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:48.908195 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:48.908195 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:48.908195 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 21:03:48.908195 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:03:48.908195 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 21:03:48.908195 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:48.908195 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 443/884] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Aug 4 21:03:48 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:48.913544 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a2054b-5425-4560-834b-cfbb9b7c2918 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:49.016022 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a2054b-5425-4560-834b-cfbb9b7c2918 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:49.063349 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2863bbf2-345f-45ce-999f-f123cd8ace25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:49.064888 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2863bbf2-345f-45ce-999f-f123cd8ace25 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:49.065401 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2863bbf2-345f-45ce-999f-f123cd8ace25 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:49.065871 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2863bbf2-345f-45ce-999f-f123cd8ace25 None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:49.066780 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2863bbf2-345f-45ce-999f-f123cd8ace25 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:49.067352 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2863bbf2-345f-45ce-999f-f123cd8ace25 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:49.068922 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2863bbf2-345f-45ce-999f-f123cd8ace25 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:49.074457 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 442/885] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Aug 4 21:03:48 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:49.080532 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59b8bc9-c281-4d8c-8589-92a84205f81a None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:49.189726 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59b8bc9-c281-4d8c-8589-92a84205f81a None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:49.225454 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c3708d-a85a-46e6-8430-5efd21f0624d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:49.226082 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c3708d-a85a-46e6-8430-5efd21f0624d None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:49.226363 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c3708d-a85a-46e6-8430-5efd21f0624d None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:49.226632 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c3708d-a85a-46e6-8430-5efd21f0624d None admin] PATH_INFO: `/v3/services` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:49.227330 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c3708d-a85a-46e6-8430-5efd21f0624d None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:49.227640 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c3708d-a85a-46e6-8430-5efd21f0624d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:49.229004 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c3708d-a85a-46e6-8430-5efd21f0624d None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:49.234530 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:49.239070 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 444/886] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Aug 4 21:03:49 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:49.240279 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2863bbf2-345f-45ce-999f-f123cd8ace25 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:49.260599 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:49.344874 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2863bbf2-345f-45ce-999f-f123cd8ace25 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:49.380564 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d689f373-2796-4583-9ef2-51a068b72263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:49.381580 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d689f373-2796-4583-9ef2-51a068b72263 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:49.382057 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d689f373-2796-4583-9ef2-51a068b72263 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:49.382551 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d689f373-2796-4583-9ef2-51a068b72263 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:49.383541 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d689f373-2796-4583-9ef2-51a068b72263 None admin] RBAC: Adding query filter params (interface=public, service_id=1374faf931ca43409dad7b9c5d80c76b, region_id=RegionOne) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:49.384044 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d689f373-2796-4583-9ef2-51a068b72263 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:49.385606 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d689f373-2796-4583-9ef2-51a068b72263 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:49.391598 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 443/887] 10.222.0.25 () {56 vars in 1251 bytes} [Thu Aug 4 21:03:49 2022] GET /identity/v3/endpoints?service_id=1374faf931ca43409dad7b9c5d80c76b&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:49.403367 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c3708d-a85a-46e6-8430-5efd21f0624d None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:49.513462 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c3708d-a85a-46e6-8430-5efd21f0624d None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:49.548095 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae0d54b-b8f2-40ce-ae83-f46b4ae7bfbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:49.549327 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae0d54b-b8f2-40ce-ae83-f46b4ae7bfbd None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:49.549612 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae0d54b-b8f2-40ce-ae83-f46b4ae7bfbd None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:49.549890 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae0d54b-b8f2-40ce-ae83-f46b4ae7bfbd None admin] PATH_INFO: `/v3/services` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:49.550547 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae0d54b-b8f2-40ce-ae83-f46b4ae7bfbd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:49.551946 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae0d54b-b8f2-40ce-ae83-f46b4ae7bfbd None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:49.558034 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 445/888] 10.222.0.25 () {56 vars in 1087 bytes} [Thu Aug 4 21:03:49 2022] GET /identity/v3/services => generated 1268 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:03:49.989176 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e25c6a-1b44-4fc2-a43c-3c25ec514b45 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:49.990173 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e25c6a-1b44-4fc2-a43c-3c25ec514b45 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:49.990655 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e25c6a-1b44-4fc2-a43c-3c25ec514b45 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:49.992485 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 444/889] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:03:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:03:49.998081 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a973f4-3a00-47fb-8ef6-e3bac1b46bf0 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:49.998889 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a973f4-3a00-47fb-8ef6-e3bac1b46bf0 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:49.999407 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a973f4-3a00-47fb-8ef6-e3bac1b46bf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:50.018748 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-f5a973f4-3a00-47fb-8ef6-e3bac1b46bf0 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:03:50.026880 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-f5a973f4-3a00-47fb-8ef6-e3bac1b46bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:03:50.081888 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:50.088080 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08495705-4e46-4c69-bbc8-9b97bbb47690 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:50.088386 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08495705-4e46-4c69-bbc8-9b97bbb47690 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:50.088652 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08495705-4e46-4c69-bbc8-9b97bbb47690 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:50.090360 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 445/891] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:03:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:03:50.090475 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 446/891] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:03:49 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 21:03:50.095968 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ceebd5-c896-4534-b61b-1e36179c2289 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:50.096448 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ceebd5-c896-4534-b61b-1e36179c2289 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:50.099613 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ceebd5-c896-4534-b61b-1e36179c2289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:50.115489 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-74ceebd5-c896-4534-b61b-1e36179c2289 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:03:50.116185 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a973f4-3a00-47fb-8ef6-e3bac1b46bf0 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:50.117208 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-74ceebd5-c896-4534-b61b-1e36179c2289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:03:50.203361 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 446/892] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:03:50 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:03:50.226248 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ceebd5-c896-4534-b61b-1e36179c2289 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:50.245630 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:50.246599 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a973f4-3a00-47fb-8ef6-e3bac1b46bf0 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:50.283603 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e00e0da-12e7-425e-b3c2-29a584c0c0bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:50.284784 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e00e0da-12e7-425e-b3c2-29a584c0c0bd None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:50.285244 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e00e0da-12e7-425e-b3c2-29a584c0c0bd None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:50.285743 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e00e0da-12e7-425e-b3c2-29a584c0c0bd None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:50.297112 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e00e0da-12e7-425e-b3c2-29a584c0c0bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:03:50.302336 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-8e00e0da-12e7-425e-b3c2-29a584c0c0bd None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 21:03:50.302336 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:50.302336 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:50.302336 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:50.302336 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:50.302336 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:50.302336 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:50.302336 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:50.302336 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:50.302336 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:50.302336 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:50.302336 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:50.302336 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:03:50.302336 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:03:50.302336 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:03:50.302336 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:03:50.303252 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:03:50.303252 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:03:50.303252 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:03:50.303252 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:03:50.303252 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:50.303252 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:50.303252 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:50.303252 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:50.303252 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:50.303252 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:50.303252 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:50.303252 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:50.303252 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:50.303252 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:50.303252 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:50.303252 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:50.304144 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:50.304144 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:50.304144 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:50.304144 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:50.304144 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:03:50.304144 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:03:50.304144 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:03:50.304144 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:03:50.304144 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:03:50.304144 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:03:50.304144 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 21:03:50.304144 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:50.311451 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 447/893] 10.222.0.25 () {56 vars in 1120 bytes} [Thu Aug 4 21:03:50 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:03:50.313120 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e00e0da-12e7-425e-b3c2-29a584c0c0bd None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:50.353353 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ceebd5-c896-4534-b61b-1e36179c2289 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:50.374153 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:50.401099 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f821585-e965-4803-b1bf-755ce82fbbf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:50.401936 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f821585-e965-4803-b1bf-755ce82fbbf8 None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:50.402310 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f821585-e965-4803-b1bf-755ce82fbbf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:50.402693 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f821585-e965-4803-b1bf-755ce82fbbf8 None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:50.403664 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f821585-e965-4803-b1bf-755ce82fbbf8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:50.405787 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f821585-e965-4803-b1bf-755ce82fbbf8 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:50.424072 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e00e0da-12e7-425e-b3c2-29a584c0c0bd None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:50.459871 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf0732c-6ccc-4ced-b325-2cccba4da359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:50.460491 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf0732c-6ccc-4ced-b325-2cccba4da359 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:50.460770 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf0732c-6ccc-4ced-b325-2cccba4da359 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:50.461040 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf0732c-6ccc-4ced-b325-2cccba4da359 None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:50.461755 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf0732c-6ccc-4ced-b325-2cccba4da359 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:50.462113 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf0732c-6ccc-4ced-b325-2cccba4da359 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:50.463537 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf0732c-6ccc-4ced-b325-2cccba4da359 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:50.471920 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 448/894] 10.222.0.25 () {56 vars in 1115 bytes} [Thu Aug 4 21:03:50 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:03:50.477132 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf0732c-6ccc-4ced-b325-2cccba4da359 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:50.511793 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 447/895] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Aug 4 21:03:50 2022] POST /identity/v3/services => generated 241 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:03:50.594520 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf0732c-6ccc-4ced-b325-2cccba4da359 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:50.613733 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:50.640061 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43421ac2-3a42-4866-a3cc-2065760fbc33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:50.640714 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43421ac2-3a42-4866-a3cc-2065760fbc33 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:50.640993 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43421ac2-3a42-4866-a3cc-2065760fbc33 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:50.641263 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43421ac2-3a42-4866-a3cc-2065760fbc33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:50.653953 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43421ac2-3a42-4866-a3cc-2065760fbc33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:50.655509 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43421ac2-3a42-4866-a3cc-2065760fbc33 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:50.671790 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-43421ac2-3a42-4866-a3cc-2065760fbc33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:50.671790 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:50.671790 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:50.671790 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:50.671790 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:50.671790 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:50.671790 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:50.671790 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:50.671790 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:50.671790 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:50.671790 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:50.671790 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:50.671790 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:03:50.671790 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:03:50.671790 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:03:50.672747 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:03:50.672747 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:50.672747 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:50.672747 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:50.672747 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:50.672747 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:50.672747 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:50.672747 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:50.672747 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:50.672747 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:50.672747 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:50.672747 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:50.672747 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:50.672747 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:50.672747 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:50.672747 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:50.672747 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:50.674191 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:03:50.674191 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:03:50.674191 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:50.674191 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:50.674191 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 449/896] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:03:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:03:50.679457 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f821585-e965-4803-b1bf-755ce82fbbf8 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:50.742356 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04aaf5c-1cef-4d88-b38d-4f3c1bb4e608 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:50.742699 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04aaf5c-1cef-4d88-b38d-4f3c1bb4e608 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:50.742905 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04aaf5c-1cef-4d88-b38d-4f3c1bb4e608 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:50.744524 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 450/897] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:03:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 21:03:50.750427 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af1dbfa-bfd7-42ee-bd24-11aa8b0a9874 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:50.750707 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af1dbfa-bfd7-42ee-bd24-11aa8b0a9874 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:50.750979 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af1dbfa-bfd7-42ee-bd24-11aa8b0a9874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:50.760659 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:50.776251 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-3af1dbfa-bfd7-42ee-bd24-11aa8b0a9874 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:03:50.776251 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-3af1dbfa-bfd7-42ee-bd24-11aa8b0a9874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:03:50.791016 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f821585-e965-4803-b1bf-755ce82fbbf8 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:50.826153 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2139993b-42cf-4175-be73-b94883869c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:50.826809 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2139993b-42cf-4175-be73-b94883869c17 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:50.827310 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2139993b-42cf-4175-be73-b94883869c17 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:50.827766 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2139993b-42cf-4175-be73-b94883869c17 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:50.828936 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2139993b-42cf-4175-be73-b94883869c17 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:50.829360 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2139993b-42cf-4175-be73-b94883869c17 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:50.830854 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2139993b-42cf-4175-be73-b94883869c17 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:50.843543 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 448/898] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:03:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:50.850921 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2139993b-42cf-4175-be73-b94883869c17 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:50.851587 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 451/899] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:03:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 21:03:50.873965 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af1dbfa-bfd7-42ee-bd24-11aa8b0a9874 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:50.973663 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2139993b-42cf-4175-be73-b94883869c17 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:50.981125 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af1dbfa-bfd7-42ee-bd24-11aa8b0a9874 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:51.007222 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:51.012224 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3859de6-3172-489b-b318-ff90facf1ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:51.012870 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3859de6-3172-489b-b318-ff90facf1ecc None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:51.013149 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3859de6-3172-489b-b318-ff90facf1ecc None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:51.013421 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3859de6-3172-489b-b318-ff90facf1ecc None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:51.014514 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:51.021377 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3859de6-3172-489b-b318-ff90facf1ecc None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:51.022833 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3859de6-3172-489b-b318-ff90facf1ecc None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:51.030852 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9445053-ae0c-49d2-a563-9c06996247fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:51.031836 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9445053-ae0c-49d2-a563-9c06996247fc None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:51.032320 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9445053-ae0c-49d2-a563-9c06996247fc None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:51.033403 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9445053-ae0c-49d2-a563-9c06996247fc None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:51.034545 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9445053-ae0c-49d2-a563-9c06996247fc None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:51.036719 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9445053-ae0c-49d2-a563-9c06996247fc None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:51.037066 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-c3859de6-3172-489b-b318-ff90facf1ecc None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 21:03:51.037066 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:51.037066 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:51.037066 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:51.037066 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:51.037066 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:51.037066 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:51.037066 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:51.037066 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:51.037066 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:51.037066 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:51.037066 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:51.037066 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 21:03:51.037066 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 21:03:51.037066 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 21:03:51.037066 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 21:03:51.037956 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:51.037956 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:51.037956 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 21:03:51.037956 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:03:51.037956 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 21:03:51.037956 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:03:51.037956 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:51.037956 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:51.037956 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:51.037956 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:51.037956 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:51.037956 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:51.037956 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:51.037956 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:51.037956 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:51.037956 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:51.037956 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:51.037956 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:51.038977 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:51.038977 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:51.038977 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 21:03:51.038977 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 21:03:51.038977 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 21:03:51.038977 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 21:03:51.038977 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 21:03:51.038977 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 21:03:51.038977 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 21:03:51.038977 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:51.047542 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 449/900] 10.222.0.25 () {56 vars in 1117 bytes} [Thu Aug 4 21:03:50 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:51.048874 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3859de6-3172-489b-b318-ff90facf1ecc None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:51.053741 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-f9445053-ae0c-49d2-a563-9c06996247fc None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 21:03:51.053741 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:51.053741 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 21:03:51.053741 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 21:03:51.053741 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 21:03:51.053741 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 21:03:51.053741 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 21:03:51.053741 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:03:51.053741 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 21:03:51.053741 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:51.053741 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 21:03:51.053741 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:51.053741 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:51.053741 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:51.053741 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:51.053741 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:51.054696 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:51.054696 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:51.054696 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:51.054696 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:51.054696 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:51.054696 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:51.054696 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:51.054696 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 21:03:51.054696 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 21:03:51.054696 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 21:03:51.054696 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 21:03:51.054696 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:51.054696 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:51.054696 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:51.054696 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:51.054696 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:51.060412 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:51.060412 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:51.060412 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:51.060412 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:51.060412 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:51.060412 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:51.060412 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:51.060412 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:51.060412 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:51.060412 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:51.060412 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:51.060412 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 21:03:51.060412 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:03:51.060412 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 21:03:51.060412 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:51.060412 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 452/901] 10.222.0.25 () {56 vars in 1132 bytes} [Thu Aug 4 21:03:50 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:51.066620 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9445053-ae0c-49d2-a563-9c06996247fc None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:51.177986 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3859de6-3172-489b-b318-ff90facf1ecc None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:51.180466 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9445053-ae0c-49d2-a563-9c06996247fc None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:51.217111 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87272419-8735-4b7e-95ed-9be7f72ae25a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:51.217789 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87272419-8735-4b7e-95ed-9be7f72ae25a None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:51.218064 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87272419-8735-4b7e-95ed-9be7f72ae25a None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:51.218339 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87272419-8735-4b7e-95ed-9be7f72ae25a None admin] PATH_INFO: `/v3/services` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:51.222164 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87272419-8735-4b7e-95ed-9be7f72ae25a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:51.222164 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87272419-8735-4b7e-95ed-9be7f72ae25a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:51.222164 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87272419-8735-4b7e-95ed-9be7f72ae25a None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:51.231568 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 453/902] 10.222.0.25 () {56 vars in 1126 bytes} [Thu Aug 4 21:03:51 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:51.233151 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87272419-8735-4b7e-95ed-9be7f72ae25a None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:51.238194 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d9e222-6dd5-4282-8c26-55a198cef7cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:51.239076 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d9e222-6dd5-4282-8c26-55a198cef7cc None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:51.239592 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d9e222-6dd5-4282-8c26-55a198cef7cc None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:51.240087 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d9e222-6dd5-4282-8c26-55a198cef7cc None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:51.241075 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d9e222-6dd5-4282-8c26-55a198cef7cc None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:51.241594 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d9e222-6dd5-4282-8c26-55a198cef7cc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:51.243903 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d9e222-6dd5-4282-8c26-55a198cef7cc None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:51.260462 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:51.273419 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 450/903] 10.222.0.25 () {56 vars in 1149 bytes} [Thu Aug 4 21:03:51 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:03:51.276441 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d9e222-6dd5-4282-8c26-55a198cef7cc None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:51.343411 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:51.350581 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87272419-8735-4b7e-95ed-9be7f72ae25a None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:51.389668 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b32b7b6-eac8-4a6e-81f2-b60a608b46ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:51.390479 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b32b7b6-eac8-4a6e-81f2-b60a608b46ba None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:51.390835 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b32b7b6-eac8-4a6e-81f2-b60a608b46ba None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:51.391201 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b32b7b6-eac8-4a6e-81f2-b60a608b46ba None admin] PATH_INFO: `/v3/services` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:51.392700 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b32b7b6-eac8-4a6e-81f2-b60a608b46ba None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:51.393090 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b32b7b6-eac8-4a6e-81f2-b60a608b46ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:51.394540 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b32b7b6-eac8-4a6e-81f2-b60a608b46ba None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:51.404796 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 454/904] 10.222.0.25 () {56 vars in 1126 bytes} [Thu Aug 4 21:03:51 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:03:51.405813 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b32b7b6-eac8-4a6e-81f2-b60a608b46ba None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:51.411349 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d9e222-6dd5-4282-8c26-55a198cef7cc None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:51.459665 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90a9476-3c42-445c-b0ba-eccb409faaf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:51.461069 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90a9476-3c42-445c-b0ba-eccb409faaf7 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:51.461778 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90a9476-3c42-445c-b0ba-eccb409faaf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:51.462229 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90a9476-3c42-445c-b0ba-eccb409faaf7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:51.513858 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:51.531457 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b32b7b6-eac8-4a6e-81f2-b60a608b46ba None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:51.545066 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90a9476-3c42-445c-b0ba-eccb409faaf7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:51.546633 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90a9476-3c42-445c-b0ba-eccb409faaf7 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:51.571248 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175a36bf-110b-4270-b4aa-1cf10e33659d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:51.572288 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175a36bf-110b-4270-b4aa-1cf10e33659d None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:51.572838 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175a36bf-110b-4270-b4aa-1cf10e33659d None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:51.573294 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175a36bf-110b-4270-b4aa-1cf10e33659d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:51.574260 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175a36bf-110b-4270-b4aa-1cf10e33659d None admin] RBAC: Adding query filter params (interface=public, service_id=d4647f5dc7d5456ab1be56b5e49fd0bf, region_id=RegionOne) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:51.574753 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175a36bf-110b-4270-b4aa-1cf10e33659d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:51.576558 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175a36bf-110b-4270-b4aa-1cf10e33659d None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:51.582833 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 455/905] 10.222.0.25 () {56 vars in 1251 bytes} [Thu Aug 4 21:03:51 2022] GET /identity/v3/endpoints?service_id=d4647f5dc7d5456ab1be56b5e49fd0bf&interface=public®ion_id=RegionOne => generated 194 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:03:51.588254 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175a36bf-110b-4270-b4aa-1cf10e33659d None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:51.602407 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:51.655822 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 451/906] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Aug 4 21:03:51 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 21:03:51.660939 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6051e525-2a39-425e-8e75-4204bc332d86 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:51.661072 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6051e525-2a39-425e-8e75-4204bc332d86 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:51.661302 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6051e525-2a39-425e-8e75-4204bc332d86 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:51.662606 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 452/907] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:03:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:03:51.663297 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6051e525-2a39-425e-8e75-4204bc332d86 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:51.711739 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175a36bf-110b-4270-b4aa-1cf10e33659d None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:51.748522 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760a0237-6667-413d-bb67-08a24ba62f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:51.749147 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a0237-6667-413d-bb67-08a24ba62f7f None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:51.749433 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a0237-6667-413d-bb67-08a24ba62f7f None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:51.749706 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a0237-6667-413d-bb67-08a24ba62f7f None admin] PATH_INFO: `/v3/services` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:51.750366 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760a0237-6667-413d-bb67-08a24ba62f7f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:51.752346 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760a0237-6667-413d-bb67-08a24ba62f7f None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:51.758879 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 456/908] 10.222.0.25 () {56 vars in 1087 bytes} [Thu Aug 4 21:03:51 2022] GET /identity/v3/services => generated 1497 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:03:51.760150 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c743f0a-4c5c-4f17-a502-749c24217081 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:51.760445 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c743f0a-4c5c-4f17-a502-749c24217081 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:51.760724 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c743f0a-4c5c-4f17-a502-749c24217081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:51.780437 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-7c743f0a-4c5c-4f17-a502-749c24217081 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:03:51.781691 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-7c743f0a-4c5c-4f17-a502-749c24217081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:03:51.789459 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6051e525-2a39-425e-8e75-4204bc332d86 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:51.794877 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:51.838832 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2e4fb9-3591-4036-a4c3-01f7984e03e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:51.840759 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2e4fb9-3591-4036-a4c3-01f7984e03e2 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:51.841283 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2e4fb9-3591-4036-a4c3-01f7984e03e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:51.841733 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2e4fb9-3591-4036-a4c3-01f7984e03e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:51.858535 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2e4fb9-3591-4036-a4c3-01f7984e03e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:51.859987 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2e4fb9-3591-4036-a4c3-01f7984e03e2 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:51.876402 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 457/909] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:03:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:03:51.876936 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-bb2e4fb9-3591-4036-a4c3-01f7984e03e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:51.876936 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:51.876936 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:51.876936 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:51.876936 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:51.876936 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:51.876936 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:51.876936 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:51.876936 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:51.876936 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:51.876936 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:51.876936 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:51.876936 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:03:51.876936 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:03:51.876936 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:03:51.877798 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:03:51.877798 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:51.877798 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:51.877798 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:51.877798 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:51.877798 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:51.877798 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:51.877798 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:51.877798 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:51.877798 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:51.877798 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:51.877798 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:51.877798 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:51.877798 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:51.877798 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:51.877798 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:51.877798 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:51.878671 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:03:51.878671 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:03:51.878671 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:51.878671 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:51.882823 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 453/910] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:03:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:51.887619 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c743f0a-4c5c-4f17-a502-749c24217081 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:51.898596 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2e4fb9-3591-4036-a4c3-01f7984e03e2 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:51.901894 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:52.010217 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c743f0a-4c5c-4f17-a502-749c24217081 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:52.035643 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2e4fb9-3591-4036-a4c3-01f7984e03e2 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:52.047157 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e512955-7ea4-4032-bdbb-804fc6f6a119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:52.049028 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e512955-7ea4-4032-bdbb-804fc6f6a119 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:52.049518 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e512955-7ea4-4032-bdbb-804fc6f6a119 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:52.050014 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e512955-7ea4-4032-bdbb-804fc6f6a119 None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:52.050926 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e512955-7ea4-4032-bdbb-804fc6f6a119 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:52.051463 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e512955-7ea4-4032-bdbb-804fc6f6a119 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:52.053068 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e512955-7ea4-4032-bdbb-804fc6f6a119 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:52.065508 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 458/911] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:03:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:52.071567 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e512955-7ea4-4032-bdbb-804fc6f6a119 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:52.076626 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:52.079936 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce68a497-0594-4197-b4af-88aab547467c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:52.080800 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce68a497-0594-4197-b4af-88aab547467c None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:52.081284 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce68a497-0594-4197-b4af-88aab547467c None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:52.081741 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce68a497-0594-4197-b4af-88aab547467c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:52.082569 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce68a497-0594-4197-b4af-88aab547467c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:52.084218 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce68a497-0594-4197-b4af-88aab547467c None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:52.093818 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-ce68a497-0594-4197-b4af-88aab547467c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 21:03:52.093818 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:52.093818 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 21:03:52.093818 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 21:03:52.093818 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 21:03:52.093818 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 21:03:52.093818 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 21:03:52.093818 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:03:52.093818 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 21:03:52.093818 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:52.093818 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 21:03:52.093818 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:52.093818 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:52.093818 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:52.093818 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:52.093818 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:52.093818 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:52.094886 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:52.094886 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:52.094886 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:52.094886 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:52.094886 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:52.094886 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:52.094886 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 21:03:52.094886 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 21:03:52.094886 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 21:03:52.094886 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 21:03:52.094886 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:52.094886 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:52.094886 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:52.094886 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:52.094886 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:52.094886 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:52.096594 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:52.096594 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:52.096594 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:52.096594 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:52.096594 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:52.096594 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:52.096594 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:52.096594 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:52.096594 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:52.096594 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:52.096594 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 21:03:52.096594 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:03:52.096594 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 21:03:52.096594 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:52.103422 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 454/912] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Aug 4 21:03:51 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:03:52.105328 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce68a497-0594-4197-b4af-88aab547467c None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:52.184089 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:52.200670 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e512955-7ea4-4032-bdbb-804fc6f6a119 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:52.225426 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce68a497-0594-4197-b4af-88aab547467c None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:52.253727 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2bf147-950d-4d82-9fd2-eb8a058987cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:52.254640 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2bf147-950d-4d82-9fd2-eb8a058987cd None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:52.254929 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2bf147-950d-4d82-9fd2-eb8a058987cd None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:52.255209 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2bf147-950d-4d82-9fd2-eb8a058987cd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:52.264024 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2bf147-950d-4d82-9fd2-eb8a058987cd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:52.265320 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6ff592-dd81-4e1f-af84-2ee313859df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:52.265525 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2bf147-950d-4d82-9fd2-eb8a058987cd None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:52.266575 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6ff592-dd81-4e1f-af84-2ee313859df9 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:52.267660 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6ff592-dd81-4e1f-af84-2ee313859df9 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:52.268280 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6ff592-dd81-4e1f-af84-2ee313859df9 None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:52.269270 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6ff592-dd81-4e1f-af84-2ee313859df9 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:52.269778 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6ff592-dd81-4e1f-af84-2ee313859df9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:52.271693 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6ff592-dd81-4e1f-af84-2ee313859df9 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:52.277370 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 455/913] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Aug 4 21:03:52 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:03:52.282003 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-9d2bf147-950d-4d82-9fd2-eb8a058987cd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:03:52.282003 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:52.282003 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:52.282003 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:52.282003 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:52.282003 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:52.282003 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:52.282003 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:52.282003 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:52.282003 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:52.282003 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:52.282003 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:52.282003 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 21:03:52.282003 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 21:03:52.282003 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 21:03:52.283592 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 21:03:52.283592 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:52.283592 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:52.283592 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:52.283592 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:52.283592 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:52.283592 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:52.283592 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:52.283592 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:52.283592 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:52.283592 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:52.283592 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:52.283592 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:52.283592 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:52.283592 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:52.283592 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:52.283592 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:52.284755 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 21:03:52.284755 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 21:03:52.284755 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 21:03:52.284755 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 21:03:52.284755 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 21:03:52.284755 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 21:03:52.284755 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:03:52.284755 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:52.285823 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 459/914] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Aug 4 21:03:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:03:52.286831 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6ff592-dd81-4e1f-af84-2ee313859df9 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:52.286940 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2bf147-950d-4d82-9fd2-eb8a058987cd None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:52.354225 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:52.393787 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2bf147-950d-4d82-9fd2-eb8a058987cd None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:52.418002 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6ff592-dd81-4e1f-af84-2ee313859df9 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:52.436217 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8070468e-f1f7-49e2-81c1-8f96e75d396a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:52.437063 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8070468e-f1f7-49e2-81c1-8f96e75d396a None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:52.437359 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8070468e-f1f7-49e2-81c1-8f96e75d396a None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:52.437639 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8070468e-f1f7-49e2-81c1-8f96e75d396a None admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:52.438408 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8070468e-f1f7-49e2-81c1-8f96e75d396a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:52.438750 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8070468e-f1f7-49e2-81c1-8f96e75d396a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:52.440256 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8070468e-f1f7-49e2-81c1-8f96e75d396a None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:52.452399 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:52.456494 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c2f306-869e-403c-a11b-8033af446d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:52.457355 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c2f306-869e-403c-a11b-8033af446d5a None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:52.457870 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c2f306-869e-403c-a11b-8033af446d5a None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:52.458368 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c2f306-869e-403c-a11b-8033af446d5a None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:52.459352 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c2f306-869e-403c-a11b-8033af446d5a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:52.459877 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c2f306-869e-403c-a11b-8033af446d5a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:52.461626 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c2f306-869e-403c-a11b-8033af446d5a None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:52.461878 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 460/915] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Aug 4 21:03:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:03:52.463734 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8070468e-f1f7-49e2-81c1-8f96e75d396a None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:52.477515 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 456/916] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Aug 4 21:03:52 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:52.478739 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c2f306-869e-403c-a11b-8033af446d5a None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:52.586696 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8070468e-f1f7-49e2-81c1-8f96e75d396a None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:52.596681 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c2f306-869e-403c-a11b-8033af446d5a None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:52.622661 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:52.631683 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c30da6c-39c8-4d45-8925-8c048521d858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:52.633807 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c30da6c-39c8-4d45-8925-8c048521d858 None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:52.633807 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c30da6c-39c8-4d45-8925-8c048521d858 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:52.633807 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c30da6c-39c8-4d45-8925-8c048521d858 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:52.634589 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c30da6c-39c8-4d45-8925-8c048521d858 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:52.635407 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b8627a-00e7-46b8-82b8-98ee49e02298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:52.636082 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b8627a-00e7-46b8-82b8-98ee49e02298 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:52.636374 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b8627a-00e7-46b8-82b8-98ee49e02298 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:52.636654 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b8627a-00e7-46b8-82b8-98ee49e02298 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:52.637429 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c30da6c-39c8-4d45-8925-8c048521d858 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:52.637546 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b8627a-00e7-46b8-82b8-98ee49e02298 None admin] RBAC: Adding query filter params (role.id=47f3f6a0aef34d25a88aed7f622b2d46, scope.project.id=17dc3d39e2c048ca9070588f5717a578, user.id=23ea380385f649628cc27e3603bc700c) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:52.637799 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b8627a-00e7-46b8-82b8-98ee49e02298 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:52.639225 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b8627a-00e7-46b8-82b8-98ee49e02298 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:52.645864 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 461/917] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Aug 4 21:03:52 2022] GET /identity/v3/role_assignments?user.id=23ea380385f649628cc27e3603bc700c&scope.project.id=17dc3d39e2c048ca9070588f5717a578&role.id=47f3f6a0aef34d25a88aed7f622b2d46 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:03:52.647185 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda61d45-11b0-4353-888b-d55516749f5d None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:52.647508 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda61d45-11b0-4353-888b-d55516749f5d None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:52.647789 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda61d45-11b0-4353-888b-d55516749f5d None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:52.651360 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 462/918] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:03:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:03:52.654602 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd74a037-02bb-4942-a004-ebe5c388a165 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:52.654898 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd74a037-02bb-4942-a004-ebe5c388a165 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:52.655181 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd74a037-02bb-4942-a004-ebe5c388a165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:52.675565 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-dd74a037-02bb-4942-a004-ebe5c388a165 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:03:52.676806 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-dd74a037-02bb-4942-a004-ebe5c388a165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:03:52.717640 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 457/919] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:03:52 2022] POST /identity/v3/endpoints => generated 346 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:03:52.770514 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 463/920] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:03:52 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 21:03:52.793705 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c30da6c-39c8-4d45-8925-8c048521d858 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:52.851934 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:52.912253 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c30da6c-39c8-4d45-8925-8c048521d858 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:52.952946 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971b0696-ce2e-429b-99c8-c2e6c0c6090a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:52.953974 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971b0696-ce2e-429b-99c8-c2e6c0c6090a None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:52.954497 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971b0696-ce2e-429b-99c8-c2e6c0c6090a None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:52.954976 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971b0696-ce2e-429b-99c8-c2e6c0c6090a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:52.955846 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971b0696-ce2e-429b-99c8-c2e6c0c6090a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:52.957439 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971b0696-ce2e-429b-99c8-c2e6c0c6090a None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:52.967363 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-971b0696-ce2e-429b-99c8-c2e6c0c6090a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 21:03:52.967363 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:52.967363 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 21:03:52.967363 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 21:03:52.967363 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 21:03:52.967363 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 21:03:52.967363 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 21:03:52.967363 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:03:52.967363 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 21:03:52.967363 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:52.967363 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 21:03:52.967363 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:52.967363 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:52.967363 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:52.967363 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:52.967363 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:52.967363 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:52.968458 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:52.968458 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:52.968458 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:52.968458 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:52.968458 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:52.968458 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:52.968458 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 21:03:52.968458 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 21:03:52.968458 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 21:03:52.968458 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 21:03:52.968458 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:52.968458 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:52.968458 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:52.968458 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:52.968458 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:52.968458 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:52.969496 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:52.969496 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:52.969496 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:52.969496 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:52.969496 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:52.969496 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:52.969496 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:52.969496 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:52.969496 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:52.969496 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:52.969496 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 21:03:52.969496 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:03:52.969496 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 21:03:52.969496 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:52.972113 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 458/921] 10.222.0.25 () {56 vars in 1126 bytes} [Thu Aug 4 21:03:52 2022] GET /identity/v3/services/object-store => generated 93 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:52.978520 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd74a037-02bb-4942-a004-ebe5c388a165 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:53.093369 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd74a037-02bb-4942-a004-ebe5c388a165 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:53.108050 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:53.130531 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32af0cdc-4433-47da-ade1-00102018b25b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:53.131198 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af0cdc-4433-47da-ade1-00102018b25b None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:53.131511 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af0cdc-4433-47da-ade1-00102018b25b None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:53.131786 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af0cdc-4433-47da-ade1-00102018b25b None admin] PATH_INFO: `/v3/services` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:53.132461 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32af0cdc-4433-47da-ade1-00102018b25b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:53.132777 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32af0cdc-4433-47da-ade1-00102018b25b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:53.134131 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32af0cdc-4433-47da-ade1-00102018b25b None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:53.139343 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 464/922] 10.222.0.25 () {56 vars in 1122 bytes} [Thu Aug 4 21:03:52 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:03:53.144753 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971b0696-ce2e-429b-99c8-c2e6c0c6090a None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:53.252652 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971b0696-ce2e-429b-99c8-c2e6c0c6090a None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:53.267794 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:53.290003 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3106ff-9204-40a7-88f5-d320fb9221a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:53.296952 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3106ff-9204-40a7-88f5-d320fb9221a6 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:53.296952 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3106ff-9204-40a7-88f5-d320fb9221a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:53.296952 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3106ff-9204-40a7-88f5-d320fb9221a6 None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:53.296952 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3106ff-9204-40a7-88f5-d320fb9221a6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:53.296952 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3106ff-9204-40a7-88f5-d320fb9221a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:53.296952 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3106ff-9204-40a7-88f5-d320fb9221a6 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:53.299204 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 459/923] 10.222.0.25 () {56 vars in 1122 bytes} [Thu Aug 4 21:03:53 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:03:53.305112 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32af0cdc-4433-47da-ade1-00102018b25b None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:53.412468 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32af0cdc-4433-47da-ade1-00102018b25b None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:53.451501 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33b7142-e6d6-4689-b243-7b3155f766f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:53.452133 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33b7142-e6d6-4689-b243-7b3155f766f5 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:53.452415 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33b7142-e6d6-4689-b243-7b3155f766f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:53.452691 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33b7142-e6d6-4689-b243-7b3155f766f5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:53.453454 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33b7142-e6d6-4689-b243-7b3155f766f5 None admin] RBAC: Adding query filter params (interface=public, service_id=42b29e257c1c4000897f8dce08c20609, region_id=RegionOne) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:53.453761 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33b7142-e6d6-4689-b243-7b3155f766f5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:53.455716 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33b7142-e6d6-4689-b243-7b3155f766f5 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:53.461304 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 465/924] 10.222.0.25 () {56 vars in 1251 bytes} [Thu Aug 4 21:03:53 2022] GET /identity/v3/endpoints?service_id=42b29e257c1c4000897f8dce08c20609&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:53.466817 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3106ff-9204-40a7-88f5-d320fb9221a6 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:53.586340 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3106ff-9204-40a7-88f5-d320fb9221a6 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:53.623343 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc996149-1b8f-43fe-843c-1c6be3153d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:53.623980 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc996149-1b8f-43fe-843c-1c6be3153d0f None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:53.624266 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc996149-1b8f-43fe-843c-1c6be3153d0f None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:53.624539 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc996149-1b8f-43fe-843c-1c6be3153d0f None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:53.625257 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc996149-1b8f-43fe-843c-1c6be3153d0f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:53.627211 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc996149-1b8f-43fe-843c-1c6be3153d0f None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:53.633701 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 460/925] 10.222.0.25 () {56 vars in 1087 bytes} [Thu Aug 4 21:03:53 2022] GET /identity/v3/services => generated 1497 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:03:53.886987 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1083661-63dc-42f9-8d1e-104fae5d81c2 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:53.887352 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1083661-63dc-42f9-8d1e-104fae5d81c2 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:53.887561 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1083661-63dc-42f9-8d1e-104fae5d81c2 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:53.889086 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 466/926] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:03:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:03:53.894782 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a6db0a-c3e0-461a-bc25-c70d99f88cda None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:53.895076 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a6db0a-c3e0-461a-bc25-c70d99f88cda None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:53.895388 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a6db0a-c3e0-461a-bc25-c70d99f88cda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:53.914710 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-f0a6db0a-c3e0-461a-bc25-c70d99f88cda None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:03:53.915999 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-f0a6db0a-c3e0-461a-bc25-c70d99f88cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:03:53.952930 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:53.990008 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 461/927] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:03:53 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:03:54.012489 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1083661-63dc-42f9-8d1e-104fae5d81c2 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:54.058127 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:54.124580 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1083661-63dc-42f9-8d1e-104fae5d81c2 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:54.160416 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1132922-1635-452f-8441-94d3c099bf62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:54.161067 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1132922-1635-452f-8441-94d3c099bf62 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:54.161351 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1132922-1635-452f-8441-94d3c099bf62 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:54.161627 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1132922-1635-452f-8441-94d3c099bf62 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:54.162260 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1132922-1635-452f-8441-94d3c099bf62 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:54.163661 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1132922-1635-452f-8441-94d3c099bf62 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:54.172767 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-f1132922-1635-452f-8441-94d3c099bf62 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 21:03:54.172767 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:54.172767 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 21:03:54.172767 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 21:03:54.172767 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 21:03:54.172767 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 21:03:54.172767 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 21:03:54.172767 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:03:54.172767 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 21:03:54.172767 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:54.172767 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 21:03:54.172767 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:54.172767 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:54.172767 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:54.172767 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:54.172767 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:54.173715 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:54.173715 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:54.173715 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:54.173715 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:54.173715 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:54.173715 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:54.173715 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:54.173715 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 21:03:54.173715 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 21:03:54.173715 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 21:03:54.173715 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 21:03:54.173715 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:54.173715 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:54.173715 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:54.173715 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:54.173715 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:54.174579 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:54.174579 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:54.174579 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:54.174579 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:54.174579 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:54.174579 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:54.174579 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:54.174579 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:54.174579 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:54.174579 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:54.174579 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:54.174579 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 21:03:54.174579 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:03:54.174579 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 21:03:54.174579 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:54.174579 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 467/928] 10.222.0.25 () {56 vars in 1132 bytes} [Thu Aug 4 21:03:54 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:54.180478 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a6db0a-c3e0-461a-bc25-c70d99f88cda None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:54.288796 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a6db0a-c3e0-461a-bc25-c70d99f88cda None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:54.328068 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ff6fb5-08e9-43e9-b360-96e9157d743b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:54.328948 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ff6fb5-08e9-43e9-b360-96e9157d743b None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:54.329944 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ff6fb5-08e9-43e9-b360-96e9157d743b None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:54.330368 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ff6fb5-08e9-43e9-b360-96e9157d743b None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:54.331202 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ff6fb5-08e9-43e9-b360-96e9157d743b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:54.331705 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ff6fb5-08e9-43e9-b360-96e9157d743b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:54.333306 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ff6fb5-08e9-43e9-b360-96e9157d743b None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:54.338990 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 462/929] 10.222.0.25 () {56 vars in 1126 bytes} [Thu Aug 4 21:03:54 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:54.352394 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1132922-1635-452f-8441-94d3c099bf62 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:54.458711 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1132922-1635-452f-8441-94d3c099bf62 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:54.467089 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:54.494159 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2cd667-d0b4-467f-827e-f5d1d5d70c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:54.494837 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2cd667-d0b4-467f-827e-f5d1d5d70c1b None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:54.495206 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2cd667-d0b4-467f-827e-f5d1d5d70c1b None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:54.495511 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2cd667-d0b4-467f-827e-f5d1d5d70c1b None admin] PATH_INFO: `/v3/services` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:54.496285 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2cd667-d0b4-467f-827e-f5d1d5d70c1b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:54.496589 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2cd667-d0b4-467f-827e-f5d1d5d70c1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:54.497947 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2cd667-d0b4-467f-827e-f5d1d5d70c1b None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:54.507657 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 468/930] 10.222.0.25 () {56 vars in 1126 bytes} [Thu Aug 4 21:03:54 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:54.509172 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ff6fb5-08e9-43e9-b360-96e9157d743b None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:54.552251 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:54.621660 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ff6fb5-08e9-43e9-b360-96e9157d743b None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:54.658320 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613d9c43-d4d4-481c-870d-229d8c16b02b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:54.659634 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613d9c43-d4d4-481c-870d-229d8c16b02b None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:54.660125 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613d9c43-d4d4-481c-870d-229d8c16b02b None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:54.660601 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613d9c43-d4d4-481c-870d-229d8c16b02b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:54.661543 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613d9c43-d4d4-481c-870d-229d8c16b02b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:54.663136 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613d9c43-d4d4-481c-870d-229d8c16b02b None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:54.767649 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 463/931] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:03:54 2022] POST /identity/v3/endpoints => generated 347 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:03:54.788962 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28b7707-3c2d-49cb-bda1-aa54405100ff None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:54.789336 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28b7707-3c2d-49cb-bda1-aa54405100ff None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:54.789515 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28b7707-3c2d-49cb-bda1-aa54405100ff None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:54.791258 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 469/932] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:03:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:03:54.797091 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097c36c5-3c1e-40ab-ab66-ec623c86c713 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:54.797385 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097c36c5-3c1e-40ab-ab66-ec623c86c713 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:54.797663 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097c36c5-3c1e-40ab-ab66-ec623c86c713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:54.817517 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-097c36c5-3c1e-40ab-ab66-ec623c86c713 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:03:54.818807 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-097c36c5-3c1e-40ab-ab66-ec623c86c713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:03:54.908429 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 464/933] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:03:54 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:03:54.912664 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9cd3de-5735-4d92-acbc-52680ee98d79 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:54.913168 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9cd3de-5735-4d92-acbc-52680ee98d79 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:54.913595 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9cd3de-5735-4d92-acbc-52680ee98d79 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:54.915112 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 470/934] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:03:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 21:03:54.923366 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c0ec1a-6487-4282-a0a6-e4714a67de3b None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:54.923366 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c0ec1a-6487-4282-a0a6-e4714a67de3b None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:54.923366 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c0ec1a-6487-4282-a0a6-e4714a67de3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:54.930275 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9cd3de-5735-4d92-acbc-52680ee98d79 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:54.940962 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-d0c0ec1a-6487-4282-a0a6-e4714a67de3b None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:03:54.942165 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-d0c0ec1a-6487-4282-a0a6-e4714a67de3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:03:54.959221 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:55.028583 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 465/935] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:03:54 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:03:55.053666 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c0ec1a-6487-4282-a0a6-e4714a67de3b None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:55.066625 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9cd3de-5735-4d92-acbc-52680ee98d79 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:55.101619 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39293bf-caa3-480f-922d-f0abb3ad69cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:55.102295 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39293bf-caa3-480f-922d-f0abb3ad69cc None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:55.102579 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39293bf-caa3-480f-922d-f0abb3ad69cc None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:55.102852 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39293bf-caa3-480f-922d-f0abb3ad69cc None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:55.113889 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39293bf-caa3-480f-922d-f0abb3ad69cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:03:55.118841 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-d39293bf-caa3-480f-922d-f0abb3ad69cc None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 21:03:55.118841 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:55.118841 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:55.118841 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:55.118841 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:55.118841 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:55.118841 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:55.118841 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:55.118841 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:55.118841 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:55.118841 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:55.118841 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:55.118841 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:03:55.118841 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:03:55.118841 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:03:55.118841 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:03:55.119824 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:03:55.119824 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:03:55.119824 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:03:55.119824 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:03:55.119824 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:55.119824 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:55.119824 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:55.119824 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:55.119824 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:55.119824 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:55.119824 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:55.119824 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:55.119824 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:55.119824 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:55.119824 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:55.119824 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:55.121072 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:55.121072 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:55.121072 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:55.121072 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:55.121072 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:03:55.121072 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:03:55.121072 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:03:55.121072 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:03:55.121072 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:03:55.121072 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:03:55.121072 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 21:03:55.121072 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:55.121072 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 471/936] 10.222.0.25 () {56 vars in 1120 bytes} [Thu Aug 4 21:03:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:03:55.125944 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39293bf-caa3-480f-922d-f0abb3ad69cc None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:55.162173 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:55.170445 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c0ec1a-6487-4282-a0a6-e4714a67de3b None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:55.216614 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79cc913-e9bc-466f-8d34-298a764e08d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:55.217373 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79cc913-e9bc-466f-8d34-298a764e08d0 None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:55.217644 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79cc913-e9bc-466f-8d34-298a764e08d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:55.217909 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79cc913-e9bc-466f-8d34-298a764e08d0 None admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:55.221419 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79cc913-e9bc-466f-8d34-298a764e08d0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:55.221419 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79cc913-e9bc-466f-8d34-298a764e08d0 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:55.222855 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: WARNING keystone.server.flask.common [None req-a79cc913-e9bc-466f-8d34-298a764e08d0 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 04 21:03:55.253033 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39293bf-caa3-480f-922d-f0abb3ad69cc None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:55.273247 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.sql.core [None req-a79cc913-e9bc-466f-8d34-298a764e08d0 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 04 21:03:55.273247 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 04 21:03:55.273247 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [parameters: {'id': '7adb02ebe84a4895b08e7108ba758143', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 04 21:03:55.273247 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80600) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 04 21:03:55.292052 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-a79cc913-e9bc-466f-8d34-298a764e08d0 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7adb02ebe84a4895b08e7108ba758143.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7adb02ebe84a4895b08e7108ba758143. Aug 04 21:03:55.292052 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:55.292052 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 04 21:03:55.292052 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 04 21:03:55.292052 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 04 21:03:55.292052 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 04 21:03:55.292052 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 04 21:03:55.292052 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application result = self._query(query) Aug 04 21:03:55.292052 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 04 21:03:55.292052 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application conn.query(q) Aug 04 21:03:55.292052 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 04 21:03:55.292052 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 04 21:03:55.292052 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 04 21:03:55.292942 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application result.read() Aug 04 21:03:55.292942 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 04 21:03:55.292942 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 04 21:03:55.292942 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 04 21:03:55.292942 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 04 21:03:55.292942 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 04 21:03:55.292942 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 04 21:03:55.292942 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 04 21:03:55.292942 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 04 21:03:55.292942 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 04 21:03:55.292942 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:55.292942 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 04 21:03:55.292942 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:55.292942 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:55.292942 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 04 21:03:55.292942 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 04 21:03:55.292942 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 04 21:03:55.292942 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 04 21:03:55.293898 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 04 21:03:55.293898 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application next(self.gen) Aug 04 21:03:55.293898 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 04 21:03:55.293898 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application yield resource Aug 04 21:03:55.293898 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 04 21:03:55.293898 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application next(self.gen) Aug 04 21:03:55.293898 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 04 21:03:55.293898 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self.session.rollback() Aug 04 21:03:55.293898 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 04 21:03:55.293898 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self.force_reraise() Aug 04 21:03:55.293898 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 04 21:03:55.293898 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise self.value Aug 04 21:03:55.293898 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 04 21:03:55.293898 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 04 21:03:55.293898 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 04 21:03:55.293898 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application session.commit() Aug 04 21:03:55.293898 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 04 21:03:55.294780 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 04 21:03:55.294780 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 04 21:03:55.294780 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._prepare_impl() Aug 04 21:03:55.294780 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 04 21:03:55.294780 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self.session.flush() Aug 04 21:03:55.294780 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 04 21:03:55.294780 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._flush(objects) Aug 04 21:03:55.294780 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 04 21:03:55.294780 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 04 21:03:55.294780 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 04 21:03:55.294780 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application compat.raise_( Aug 04 21:03:55.294780 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 04 21:03:55.294780 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception Aug 04 21:03:55.294780 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 04 21:03:55.294780 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application flush_context.execute() Aug 04 21:03:55.294780 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 04 21:03:55.294780 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rec.execute(self) Aug 04 21:03:55.295714 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 04 21:03:55.295714 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 04 21:03:55.295714 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 04 21:03:55.295714 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 04 21:03:55.295714 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 04 21:03:55.295714 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 04 21:03:55.295714 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 04 21:03:55.295714 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 04 21:03:55.295714 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 04 21:03:55.295714 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 04 21:03:55.295714 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 04 21:03:55.295714 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 04 21:03:55.295714 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 04 21:03:55.295714 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 04 21:03:55.295714 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 04 21:03:55.298886 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 04 21:03:55.298886 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 04 21:03:55.298886 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception Aug 04 21:03:55.298886 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 04 21:03:55.298886 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 04 21:03:55.298886 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 04 21:03:55.298886 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 04 21:03:55.298886 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 04 21:03:55.298886 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application result = self._query(query) Aug 04 21:03:55.298886 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 04 21:03:55.298886 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application conn.query(q) Aug 04 21:03:55.298886 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 04 21:03:55.298886 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 04 21:03:55.298886 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 04 21:03:55.298886 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application result.read() Aug 04 21:03:55.298886 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 04 21:03:55.298886 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 04 21:03:55.301318 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 04 21:03:55.301318 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 04 21:03:55.301318 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 04 21:03:55.301318 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 04 21:03:55.301318 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 04 21:03:55.301318 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 04 21:03:55.301318 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 04 21:03:55.301318 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 04 21:03:55.301318 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [parameters: {'id': '7adb02ebe84a4895b08e7108ba758143', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 04 21:03:55.301318 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 04 21:03:55.301318 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:55.301318 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 21:03:55.301318 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:55.301318 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:55.301318 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 04 21:03:55.303211 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 04 21:03:55.303211 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 04 21:03:55.303211 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 04 21:03:55.303211 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 04 21:03:55.303211 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:55.303211 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 21:03:55.303211 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:55.303211 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:55.303211 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:55.303211 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:55.303211 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:55.303211 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:55.303211 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:55.303211 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:55.303211 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:55.303211 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:55.303211 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:55.306904 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:55.306904 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 04 21:03:55.306904 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 04 21:03:55.306904 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:55.306904 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:55.306904 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 04 21:03:55.306904 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 04 21:03:55.306904 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7adb02ebe84a4895b08e7108ba758143. Aug 04 21:03:55.306904 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:55.306904 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 466/937] 10.222.0.25 () {60 vars in 1140 bytes} [Thu Aug 4 21:03:55 2022] POST /identity/v3/projects => generated 262 bytes in 243 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:03:55.306904 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79cc913-e9bc-466f-8d34-298a764e08d0 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:55.308848 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:55.308967 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4bad94-4b1f-435e-865e-8af554b43dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:55.309896 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4bad94-4b1f-435e-865e-8af554b43dda None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:55.312796 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4bad94-4b1f-435e-865e-8af554b43dda None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:55.312796 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4bad94-4b1f-435e-865e-8af554b43dda None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:55.312796 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4bad94-4b1f-435e-865e-8af554b43dda None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:55.345265 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4bad94-4b1f-435e-865e-8af554b43dda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:55.345265 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4bad94-4b1f-435e-865e-8af554b43dda None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:55.345265 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 472/938] 10.222.0.25 () {56 vars in 1115 bytes} [Thu Aug 4 21:03:55 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:03:55.345265 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4bad94-4b1f-435e-865e-8af554b43dda None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:55.431045 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79cc913-e9bc-466f-8d34-298a764e08d0 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:55.455358 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:55.467196 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4bad94-4b1f-435e-865e-8af554b43dda None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:55.473425 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63744931-98c3-413a-9fcf-5c24da2f870f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:55.474296 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63744931-98c3-413a-9fcf-5c24da2f870f None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:55.474755 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63744931-98c3-413a-9fcf-5c24da2f870f None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:55.475173 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63744931-98c3-413a-9fcf-5c24da2f870f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:55.484808 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63744931-98c3-413a-9fcf-5c24da2f870f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:55.487789 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63744931-98c3-413a-9fcf-5c24da2f870f None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:55.503894 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-63744931-98c3-413a-9fcf-5c24da2f870f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:03:55.503894 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:55.503894 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:55.503894 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:55.503894 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:55.503894 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:55.503894 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:55.503894 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:55.503894 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:55.503894 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:55.503894 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:55.503894 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:55.503894 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 21:03:55.503894 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 21:03:55.503894 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 21:03:55.504964 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 21:03:55.504964 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:55.504964 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:55.504964 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:55.504964 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:55.504964 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:55.504964 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:55.504964 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:55.504964 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:55.504964 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:55.504964 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:55.504964 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:55.504964 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:55.504964 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:55.504964 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:55.504964 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:55.504964 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:55.506057 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 21:03:55.506057 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 21:03:55.506057 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 21:03:55.506057 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 21:03:55.506057 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 21:03:55.506057 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 21:03:55.506057 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:03:55.506057 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:55.508501 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 467/939] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Aug 4 21:03:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:03:55.513988 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2799e14a-0b77-4818-b76b-517248c6ac9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:55.514613 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63744931-98c3-413a-9fcf-5c24da2f870f None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:55.514721 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2799e14a-0b77-4818-b76b-517248c6ac9a None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:55.514926 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2799e14a-0b77-4818-b76b-517248c6ac9a None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:55.515260 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2799e14a-0b77-4818-b76b-517248c6ac9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:55.532112 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2799e14a-0b77-4818-b76b-517248c6ac9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:55.536369 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2799e14a-0b77-4818-b76b-517248c6ac9a None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:55.562614 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-2799e14a-0b77-4818-b76b-517248c6ac9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:55.562614 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:55.562614 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:55.562614 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:55.562614 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:55.562614 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:55.562614 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:55.562614 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:55.562614 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:55.562614 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:55.562614 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:55.562614 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:55.562614 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:03:55.562614 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:03:55.562614 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:03:55.563587 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:03:55.563587 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:55.563587 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:55.563587 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:55.563587 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:55.563587 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:55.563587 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:55.563587 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:55.563587 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:55.563587 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:55.563587 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:55.563587 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:55.563587 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:55.563587 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:55.563587 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:55.563587 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:55.563587 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:55.565001 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:03:55.565001 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:03:55.565001 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:55.565001 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:55.572031 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 473/940] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:03:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:55.572642 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2799e14a-0b77-4818-b76b-517248c6ac9a None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:55.604401 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:55.655375 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63744931-98c3-413a-9fcf-5c24da2f870f None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:55.685138 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2799e14a-0b77-4818-b76b-517248c6ac9a None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:55.698501 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a236e12-d03d-4d70-acdd-c87454441a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:55.699601 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a236e12-d03d-4d70-acdd-c87454441a9b None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:55.700436 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a236e12-d03d-4d70-acdd-c87454441a9b None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:55.700924 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a236e12-d03d-4d70-acdd-c87454441a9b None admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:55.702086 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a236e12-d03d-4d70-acdd-c87454441a9b None admin] RBAC: Adding query filter params (name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:55.702777 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a236e12-d03d-4d70-acdd-c87454441a9b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:55.705385 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a236e12-d03d-4d70-acdd-c87454441a9b None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:55.725974 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa45d70-dbf4-4227-8dd0-47c8df33d384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:55.726923 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa45d70-dbf4-4227-8dd0-47c8df33d384 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:55.727440 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa45d70-dbf4-4227-8dd0-47c8df33d384 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:55.727939 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa45d70-dbf4-4227-8dd0-47c8df33d384 None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:55.728823 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa45d70-dbf4-4227-8dd0-47c8df33d384 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:55.729336 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa45d70-dbf4-4227-8dd0-47c8df33d384 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:55.730893 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa45d70-dbf4-4227-8dd0-47c8df33d384 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:55.735517 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 468/941] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Aug 4 21:03:55 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:03:55.745418 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 474/942] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:03:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:55.752484 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a236e12-d03d-4d70-acdd-c87454441a9b None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:55.808124 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a09a03b-2a20-4d4a-be70-6eec4cb21e3f None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:55.808879 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a09a03b-2a20-4d4a-be70-6eec4cb21e3f None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:55.809353 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a09a03b-2a20-4d4a-be70-6eec4cb21e3f None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:55.811937 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 475/943] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:03:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 21:03:55.817880 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c633b7-771c-4589-9d67-4a4b8a074e3e None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:55.818222 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c633b7-771c-4589-9d67-4a4b8a074e3e None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:55.818595 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c633b7-771c-4589-9d67-4a4b8a074e3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:55.836123 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:55.845977 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-38c633b7-771c-4589-9d67-4a4b8a074e3e None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:03:55.859875 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-38c633b7-771c-4589-9d67-4a4b8a074e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:03:55.888857 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a236e12-d03d-4d70-acdd-c87454441a9b None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:55.918602 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:55.932979 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70873aca-d1c6-4e58-bf5f-7ab6676b9d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:55.933711 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70873aca-d1c6-4e58-bf5f-7ab6676b9d75 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:55.945557 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70873aca-d1c6-4e58-bf5f-7ab6676b9d75 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:55.945557 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70873aca-d1c6-4e58-bf5f-7ab6676b9d75 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:55.945557 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70873aca-d1c6-4e58-bf5f-7ab6676b9d75 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:55.945557 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70873aca-d1c6-4e58-bf5f-7ab6676b9d75 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:55.951539 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 476/944] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:03:55 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 21:03:55.957524 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-70873aca-d1c6-4e58-bf5f-7ab6676b9d75 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 21:03:55.957524 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:55.957524 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:55.957524 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:55.957524 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:55.957524 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:55.957524 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:55.957524 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:55.957524 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:55.957524 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:55.957524 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:55.957524 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:55.957524 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 21:03:55.957524 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 21:03:55.957524 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 21:03:55.957524 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 21:03:55.958654 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:55.958654 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:55.958654 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 21:03:55.958654 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:03:55.958654 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 21:03:55.958654 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:03:55.958654 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:55.958654 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:55.958654 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:55.958654 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:55.958654 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:55.958654 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:55.958654 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:55.958654 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:55.958654 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:55.958654 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:55.958654 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:55.958654 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:55.959868 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:55.959868 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:55.959868 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 21:03:55.959868 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 21:03:55.959868 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 21:03:55.959868 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 21:03:55.959868 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 21:03:55.959868 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 21:03:55.959868 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 21:03:55.959868 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:55.960727 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 469/945] 10.222.0.25 () {56 vars in 1117 bytes} [Thu Aug 4 21:03:55 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:55.966842 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c633b7-771c-4589-9d67-4a4b8a074e3e None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:55.974933 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70873aca-d1c6-4e58-bf5f-7ab6676b9d75 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:56.088532 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70873aca-d1c6-4e58-bf5f-7ab6676b9d75 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:56.106713 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c633b7-771c-4589-9d67-4a4b8a074e3e None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:56.134794 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9208993-a2a2-4a83-9122-25be20d0f719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:56.136144 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9208993-a2a2-4a83-9122-25be20d0f719 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:56.136892 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9208993-a2a2-4a83-9122-25be20d0f719 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:56.137801 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9208993-a2a2-4a83-9122-25be20d0f719 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:56.138912 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9208993-a2a2-4a83-9122-25be20d0f719 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:56.140932 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9208993-a2a2-4a83-9122-25be20d0f719 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:56.144022 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eca14a-3055-4d6a-af73-cf52d2a4e3a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:56.145272 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eca14a-3055-4d6a-af73-cf52d2a4e3a5 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:56.145813 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eca14a-3055-4d6a-af73-cf52d2a4e3a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:56.146432 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eca14a-3055-4d6a-af73-cf52d2a4e3a5 None admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:56.147946 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eca14a-3055-4d6a-af73-cf52d2a4e3a5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:56.148363 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eca14a-3055-4d6a-af73-cf52d2a4e3a5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:56.149964 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eca14a-3055-4d6a-af73-cf52d2a4e3a5 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:56.152018 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:56.155106 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-b9208993-a2a2-4a83-9122-25be20d0f719 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 21:03:56.155106 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:56.155106 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 21:03:56.155106 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 21:03:56.155106 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 21:03:56.155106 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 21:03:56.155106 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 21:03:56.155106 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:03:56.155106 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 21:03:56.155106 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:56.155106 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 21:03:56.155106 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:56.155106 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:56.155106 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:56.155106 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:56.155106 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:56.155106 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:56.156309 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:56.156309 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:56.156309 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:56.156309 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:56.156309 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:56.156309 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:56.156309 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 21:03:56.156309 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 21:03:56.156309 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 21:03:56.156309 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 21:03:56.156309 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:56.156309 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:56.156309 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:56.156309 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:56.156309 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:56.156309 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:56.157376 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:56.157376 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:56.157376 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:56.157376 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:56.157376 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:56.157376 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:56.157376 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:56.157376 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:56.157376 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:56.157376 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:56.157376 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 21:03:56.157376 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:03:56.157376 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 21:03:56.157376 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:56.160815 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 470/946] 10.222.0.25 () {56 vars in 1126 bytes} [Thu Aug 4 21:03:55 2022] GET /identity/v3/services/object-store => generated 93 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:03:56.174638 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 477/947] 10.222.0.25 () {56 vars in 1149 bytes} [Thu Aug 4 21:03:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:56.179301 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9208993-a2a2-4a83-9122-25be20d0f719 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:56.181491 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eca14a-3055-4d6a-af73-cf52d2a4e3a5 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:56.239435 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:56.299789 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eca14a-3055-4d6a-af73-cf52d2a4e3a5 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:56.309262 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9208993-a2a2-4a83-9122-25be20d0f719 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:56.349044 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15865f9b-4580-4fa0-8ebb-61c328804dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:56.349625 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97df4011-f0cb-43e4-a03c-f0e7663db7dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:56.349771 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15865f9b-4580-4fa0-8ebb-61c328804dbb None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:56.350113 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15865f9b-4580-4fa0-8ebb-61c328804dbb None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:56.350615 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15865f9b-4580-4fa0-8ebb-61c328804dbb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:56.351152 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97df4011-f0cb-43e4-a03c-f0e7663db7dd None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:56.351601 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97df4011-f0cb-43e4-a03c-f0e7663db7dd None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:56.352060 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97df4011-f0cb-43e4-a03c-f0e7663db7dd None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:56.352895 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97df4011-f0cb-43e4-a03c-f0e7663db7dd None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:56.353343 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97df4011-f0cb-43e4-a03c-f0e7663db7dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:56.354939 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97df4011-f0cb-43e4-a03c-f0e7663db7dd None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:56.365056 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15865f9b-4580-4fa0-8ebb-61c328804dbb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:56.365056 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15865f9b-4580-4fa0-8ebb-61c328804dbb None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:56.365681 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 471/948] 10.222.0.25 () {56 vars in 1122 bytes} [Thu Aug 4 21:03:56 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:56.366696 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97df4011-f0cb-43e4-a03c-f0e7663db7dd None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:56.385109 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-15865f9b-4580-4fa0-8ebb-61c328804dbb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:56.385109 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:56.385109 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:56.385109 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:56.385109 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:56.385109 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:56.385109 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:56.385109 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:56.385109 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:56.385109 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:56.385109 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:56.385109 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:56.385109 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:03:56.385109 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:03:56.385109 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:03:56.386198 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:03:56.386198 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:56.386198 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:56.386198 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:56.386198 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:56.386198 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:56.386198 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:56.386198 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:56.386198 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:56.386198 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:56.386198 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:56.386198 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:56.386198 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:56.386198 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:56.386198 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:56.386198 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:56.386198 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:56.387124 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:03:56.387124 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:03:56.387124 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:03:56.387124 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:56.391396 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 478/949] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:03:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:56.393600 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15865f9b-4580-4fa0-8ebb-61c328804dbb None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:56.443855 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:56.503503 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15865f9b-4580-4fa0-8ebb-61c328804dbb None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:56.509713 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97df4011-f0cb-43e4-a03c-f0e7663db7dd None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:56.526707 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:56.548423 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8be36b-a036-4109-b9ab-ff2ebcfc8d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:56.549134 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8be36b-a036-4109-b9ab-ff2ebcfc8d74 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:56.549414 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8be36b-a036-4109-b9ab-ff2ebcfc8d74 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:56.549682 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8be36b-a036-4109-b9ab-ff2ebcfc8d74 None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:56.550363 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8be36b-a036-4109-b9ab-ff2ebcfc8d74 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:56.550663 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8be36b-a036-4109-b9ab-ff2ebcfc8d74 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:56.550865 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ce4056-f127-4540-a27a-f63ebfa15acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:56.551729 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce4056-f127-4540-a27a-f63ebfa15acd None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:56.552212 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8be36b-a036-4109-b9ab-ff2ebcfc8d74 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:56.552319 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce4056-f127-4540-a27a-f63ebfa15acd None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:56.552801 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce4056-f127-4540-a27a-f63ebfa15acd None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:56.553913 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ce4056-f127-4540-a27a-f63ebfa15acd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:56.554782 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ce4056-f127-4540-a27a-f63ebfa15acd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:56.556679 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ce4056-f127-4540-a27a-f63ebfa15acd None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:56.562582 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 472/950] 10.222.0.25 () {56 vars in 1122 bytes} [Thu Aug 4 21:03:56 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:56.563471 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8be36b-a036-4109-b9ab-ff2ebcfc8d74 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:56.578941 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 479/951] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:03:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:03:56.581173 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ce4056-f127-4540-a27a-f63ebfa15acd None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:56.674463 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8be36b-a036-4109-b9ab-ff2ebcfc8d74 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:56.713183 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a1db52-a39c-4a58-bedb-d987286e169e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:56.713837 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a1db52-a39c-4a58-bedb-d987286e169e None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:56.714197 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a1db52-a39c-4a58-bedb-d987286e169e None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:56.714432 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a1db52-a39c-4a58-bedb-d987286e169e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:56.715154 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a1db52-a39c-4a58-bedb-d987286e169e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:56.716551 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a1db52-a39c-4a58-bedb-d987286e169e None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:56.720477 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ce4056-f127-4540-a27a-f63ebfa15acd None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:56.724083 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:56.757428 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2bd330-9256-4fc0-bfda-75863fbf35dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:56.758073 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2bd330-9256-4fc0-bfda-75863fbf35dd None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:56.758358 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2bd330-9256-4fc0-bfda-75863fbf35dd None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:56.758715 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2bd330-9256-4fc0-bfda-75863fbf35dd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:56.765727 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2bd330-9256-4fc0-bfda-75863fbf35dd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:56.767743 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2bd330-9256-4fc0-bfda-75863fbf35dd None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:56.779117 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-ed2bd330-9256-4fc0-bfda-75863fbf35dd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:03:56.779117 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:56.779117 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:56.779117 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:56.779117 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:56.779117 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:56.779117 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:56.779117 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:56.779117 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:56.779117 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:56.779117 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:56.779117 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:56.779117 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 21:03:56.779117 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 21:03:56.779117 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 21:03:56.782743 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 21:03:56.782743 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:56.782743 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:56.782743 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:56.782743 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:56.782743 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:56.782743 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:56.782743 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:56.782743 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:56.782743 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:56.782743 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:56.782743 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:56.782743 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:56.782743 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:56.782743 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:56.782743 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:56.782743 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:56.783754 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 21:03:56.783754 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 21:03:56.783754 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 21:03:56.783754 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 21:03:56.783754 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 21:03:56.783754 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 21:03:56.783754 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:03:56.783754 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:56.783754 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 480/952] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Aug 4 21:03:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:56.786123 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2bd330-9256-4fc0-bfda-75863fbf35dd None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:56.815411 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 473/953] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:03:56 2022] POST /identity/v3/endpoints => generated 349 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:03:56.893874 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0050919-9b9c-42d1-aa9f-39124abdfcb2 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:56.894197 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0050919-9b9c-42d1-aa9f-39124abdfcb2 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:56.894424 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0050919-9b9c-42d1-aa9f-39124abdfcb2 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:56.897115 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 474/954] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:03:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 21:03:56.902766 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8981ca44-6f0e-454b-9a5a-07e4ffc3e541 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:56.903089 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8981ca44-6f0e-454b-9a5a-07e4ffc3e541 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:56.903353 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8981ca44-6f0e-454b-9a5a-07e4ffc3e541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:56.922193 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2bd330-9256-4fc0-bfda-75863fbf35dd None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:56.926505 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-8981ca44-6f0e-454b-9a5a-07e4ffc3e541 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:03:56.928387 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-8981ca44-6f0e-454b-9a5a-07e4ffc3e541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:03:56.970239 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:56.973287 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfb9b87-3a93-4c07-aae9-900df534afac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:56.974232 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfb9b87-3a93-4c07-aae9-900df534afac None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:56.974718 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfb9b87-3a93-4c07-aae9-900df534afac None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:56.975197 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfb9b87-3a93-4c07-aae9-900df534afac None admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:56.976211 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfb9b87-3a93-4c07-aae9-900df534afac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:56.976726 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfb9b87-3a93-4c07-aae9-900df534afac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:56.978575 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfb9b87-3a93-4c07-aae9-900df534afac None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:56.993891 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 481/955] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Aug 4 21:03:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:03:57.000804 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfb9b87-3a93-4c07-aae9-900df534afac None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:57.026388 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 475/956] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:03:56 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:03:57.030954 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:57.050830 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8981ca44-6f0e-454b-9a5a-07e4ffc3e541 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:57.127875 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfb9b87-3a93-4c07-aae9-900df534afac None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:57.162127 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8981ca44-6f0e-454b-9a5a-07e4ffc3e541 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:57.167376 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c76eb4-0790-41d4-b1c9-4ccd3457fd46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:57.168073 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c76eb4-0790-41d4-b1c9-4ccd3457fd46 None admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:57.168381 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c76eb4-0790-41d4-b1c9-4ccd3457fd46 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:57.168655 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c76eb4-0790-41d4-b1c9-4ccd3457fd46 None admin] PATH_INFO: `/v3/projects/17dc3d39e2c048ca9070588f5717a578/users/23ea380385f649628cc27e3603bc700c/roles/47f3f6a0aef34d25a88aed7f622b2d46` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:57.170990 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c76eb4-0790-41d4-b1c9-4ccd3457fd46 None admin] RBAC: Authorizing `identity:create_grant(project_id=17dc3d39e2c048ca9070588f5717a578, user_id=23ea380385f649628cc27e3603bc700c, role_id=47f3f6a0aef34d25a88aed7f622b2d46)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:57.172600 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c76eb4-0790-41d4-b1c9-4ccd3457fd46 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:57.197548 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313524b1-6674-4dd9-87f1-9f411943d901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:57.198106 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313524b1-6674-4dd9-87f1-9f411943d901 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:57.198383 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313524b1-6674-4dd9-87f1-9f411943d901 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:57.198646 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313524b1-6674-4dd9-87f1-9f411943d901 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:57.199397 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313524b1-6674-4dd9-87f1-9f411943d901 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:57.200873 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313524b1-6674-4dd9-87f1-9f411943d901 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:57.210061 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-313524b1-6674-4dd9-87f1-9f411943d901 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 21:03:57.210061 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:57.210061 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 21:03:57.210061 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 21:03:57.210061 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 21:03:57.210061 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 21:03:57.210061 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 21:03:57.210061 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:03:57.210061 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 21:03:57.210061 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:57.210061 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 21:03:57.210061 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:57.210061 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:57.210061 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:57.210061 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:57.210061 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:57.210061 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:57.211196 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:57.211196 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:57.211196 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:57.211196 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:57.211196 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:57.211196 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:57.211196 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 21:03:57.211196 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 21:03:57.211196 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 21:03:57.211196 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 21:03:57.211196 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:57.211196 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:57.211196 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:57.211196 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:57.211196 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:57.211196 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:57.217772 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:57.217772 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:57.217772 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:57.217772 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:57.217772 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:57.217772 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:57.217772 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:57.217772 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:57.217772 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:57.217772 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:57.217772 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 21:03:57.217772 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:03:57.217772 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 21:03:57.217772 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:57.217772 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 476/957] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Aug 4 21:03:57 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:57.219422 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313524b1-6674-4dd9-87f1-9f411943d901 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:57.238901 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 482/958] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Aug 4 21:03:56 2022] PUT /identity/v3/projects/17dc3d39e2c048ca9070588f5717a578/users/23ea380385f649628cc27e3603bc700c/roles/47f3f6a0aef34d25a88aed7f622b2d46 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:03:57.259420 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:57.343208 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313524b1-6674-4dd9-87f1-9f411943d901 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:57.381453 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffda712-974c-441b-8037-90991c29bcc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:57.382339 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffda712-974c-441b-8037-90991c29bcc7 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:57.382793 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffda712-974c-441b-8037-90991c29bcc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:57.383211 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffda712-974c-441b-8037-90991c29bcc7 None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:57.384086 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffda712-974c-441b-8037-90991c29bcc7 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:57.384535 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffda712-974c-441b-8037-90991c29bcc7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:57.386084 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffda712-974c-441b-8037-90991c29bcc7 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:57.394213 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 477/959] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Aug 4 21:03:57 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:03:57.403237 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c76eb4-0790-41d4-b1c9-4ccd3457fd46 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:57.505415 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:57.530958 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c76eb4-0790-41d4-b1c9-4ccd3457fd46 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:57.574490 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c45b10e-cc8d-4069-8534-f00f190b7881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:57.575126 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c45b10e-cc8d-4069-8534-f00f190b7881 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:57.575442 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c45b10e-cc8d-4069-8534-f00f190b7881 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:57.575720 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c45b10e-cc8d-4069-8534-f00f190b7881 None admin] PATH_INFO: `/v3/services` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:57.576415 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c45b10e-cc8d-4069-8534-f00f190b7881 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:57.576723 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c45b10e-cc8d-4069-8534-f00f190b7881 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:57.578088 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c45b10e-cc8d-4069-8534-f00f190b7881 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:57.584404 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 483/960] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Aug 4 21:03:57 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:57.944396 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a07c1f-e8ff-415c-a0d3-d9b845dcca10 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:57.944753 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a07c1f-e8ff-415c-a0d3-d9b845dcca10 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:57.945069 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a07c1f-e8ff-415c-a0d3-d9b845dcca10 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:57.946729 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 478/961] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:03:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:03:57.951990 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad82e404-b9d5-4253-97ed-4101eb8ae0e0 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:57.952287 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad82e404-b9d5-4253-97ed-4101eb8ae0e0 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:57.952568 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad82e404-b9d5-4253-97ed-4101eb8ae0e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:57.971136 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-ad82e404-b9d5-4253-97ed-4101eb8ae0e0 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:03:57.972365 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-ad82e404-b9d5-4253-97ed-4101eb8ae0e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:03:58.043811 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 484/962] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:03:57 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:03:58.068432 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a07c1f-e8ff-415c-a0d3-d9b845dcca10 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:58.162191 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:58.172255 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a07c1f-e8ff-415c-a0d3-d9b845dcca10 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:58.207848 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5480e370-bb5f-4fd6-a30b-956bd80d09bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:58.208401 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5480e370-bb5f-4fd6-a30b-956bd80d09bd None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:58.208682 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5480e370-bb5f-4fd6-a30b-956bd80d09bd None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:58.208956 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5480e370-bb5f-4fd6-a30b-956bd80d09bd None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:58.209634 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5480e370-bb5f-4fd6-a30b-956bd80d09bd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:58.211060 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5480e370-bb5f-4fd6-a30b-956bd80d09bd None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:58.267921 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.sql.core [None req-5480e370-bb5f-4fd6-a30b-956bd80d09bd None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 04 21:03:58.267921 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 04 21:03:58.267921 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [parameters: {'user_id': '1406fe9391464d5db498c478674f7efd', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 04 21:03:58.267921 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80600) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 04 21:03:58.275204 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-5480e370-bb5f-4fd6-a30b-956bd80d09bd None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 04 21:03:58.275204 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:58.275204 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 04 21:03:58.275204 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 04 21:03:58.275204 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 04 21:03:58.275204 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 04 21:03:58.275204 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 04 21:03:58.275204 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application result = self._query(query) Aug 04 21:03:58.275204 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 04 21:03:58.275204 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application conn.query(q) Aug 04 21:03:58.275204 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 04 21:03:58.275204 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 04 21:03:58.275204 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 04 21:03:58.275204 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application result.read() Aug 04 21:03:58.275204 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 04 21:03:58.280379 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 04 21:03:58.280379 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 04 21:03:58.280379 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 04 21:03:58.280379 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 04 21:03:58.280379 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 04 21:03:58.280379 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 04 21:03:58.280379 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 04 21:03:58.280379 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 04 21:03:58.280379 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:58.280379 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 04 21:03:58.280379 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:58.280379 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:58.280379 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 04 21:03:58.280379 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 04 21:03:58.280379 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 04 21:03:58.280379 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 04 21:03:58.280379 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 04 21:03:58.280379 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application next(self.gen) Aug 04 21:03:58.281423 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 04 21:03:58.281423 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application yield resource Aug 04 21:03:58.281423 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 04 21:03:58.281423 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application next(self.gen) Aug 04 21:03:58.281423 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 04 21:03:58.281423 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self.session.rollback() Aug 04 21:03:58.281423 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 04 21:03:58.281423 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self.force_reraise() Aug 04 21:03:58.281423 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 04 21:03:58.281423 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise self.value Aug 04 21:03:58.281423 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 04 21:03:58.281423 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 04 21:03:58.281423 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 04 21:03:58.281423 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application session.commit() Aug 04 21:03:58.281423 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 04 21:03:58.281423 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 04 21:03:58.281423 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 04 21:03:58.282381 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._prepare_impl() Aug 04 21:03:58.282381 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 04 21:03:58.282381 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self.session.flush() Aug 04 21:03:58.282381 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 04 21:03:58.282381 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._flush(objects) Aug 04 21:03:58.282381 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 04 21:03:58.282381 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 04 21:03:58.282381 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 04 21:03:58.282381 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application compat.raise_( Aug 04 21:03:58.282381 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 04 21:03:58.282381 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception Aug 04 21:03:58.282381 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 04 21:03:58.282381 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application flush_context.execute() Aug 04 21:03:58.282381 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 04 21:03:58.282381 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rec.execute(self) Aug 04 21:03:58.282381 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 04 21:03:58.282381 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 04 21:03:58.283504 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 04 21:03:58.283504 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 04 21:03:58.283504 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 04 21:03:58.283504 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 04 21:03:58.283504 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 04 21:03:58.283504 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 04 21:03:58.283504 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 04 21:03:58.283504 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 04 21:03:58.283504 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 04 21:03:58.283504 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 04 21:03:58.283504 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 04 21:03:58.283504 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 04 21:03:58.283504 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 04 21:03:58.283504 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 04 21:03:58.283504 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 04 21:03:58.283504 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception Aug 04 21:03:58.284777 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 04 21:03:58.284777 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 04 21:03:58.284777 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 04 21:03:58.284777 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 04 21:03:58.284777 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 04 21:03:58.284777 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application result = self._query(query) Aug 04 21:03:58.284777 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 04 21:03:58.284777 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application conn.query(q) Aug 04 21:03:58.284777 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 04 21:03:58.284777 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 04 21:03:58.284777 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 04 21:03:58.284777 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application result.read() Aug 04 21:03:58.284777 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 04 21:03:58.284777 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 04 21:03:58.284777 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 04 21:03:58.284777 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 04 21:03:58.285688 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 04 21:03:58.285688 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 04 21:03:58.285688 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 04 21:03:58.285688 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 04 21:03:58.285688 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 04 21:03:58.285688 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 04 21:03:58.285688 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [parameters: {'user_id': '1406fe9391464d5db498c478674f7efd', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 04 21:03:58.285688 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 04 21:03:58.285688 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:58.285688 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 21:03:58.285688 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:58.285688 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:58.285688 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:58.285688 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:58.285688 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:58.286530 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:58.286530 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:58.286530 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:58.286530 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:58.286530 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:58.286530 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:58.286530 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:58.286530 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 04 21:03:58.286530 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 04 21:03:58.286530 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:58.286530 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:58.286530 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 21:03:58.286530 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:03:58.286530 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 21:03:58.286530 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:03:58.286530 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 04 21:03:58.286530 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 04 21:03:58.287608 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 04 21:03:58.287608 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 04 21:03:58.287608 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 04 21:03:58.287608 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 04 21:03:58.287608 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 04 21:03:58.287608 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:03:58.287608 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 479/963] 10.222.0.25 () {60 vars in 1131 bytes} [Thu Aug 4 21:03:58 2022] POST /identity/v3/users => generated 160 bytes in 209 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:58.291075 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad82e404-b9d5-4253-97ed-4101eb8ae0e0 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:58.371736 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:58.408030 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad82e404-b9d5-4253-97ed-4101eb8ae0e0 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:58.442176 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dc6280-ee5e-4b23-869c-692b54e4d7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:58.442961 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dc6280-ee5e-4b23-869c-692b54e4d7ea None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:58.443390 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dc6280-ee5e-4b23-869c-692b54e4d7ea None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:58.443804 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dc6280-ee5e-4b23-869c-692b54e4d7ea None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:58.451601 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dc6280-ee5e-4b23-869c-692b54e4d7ea None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:58.453169 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dc6280-ee5e-4b23-869c-692b54e4d7ea None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:58.465297 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-86dc6280-ee5e-4b23-869c-692b54e4d7ea None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 21:03:58.465297 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:58.465297 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:58.465297 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:58.465297 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:58.465297 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:58.465297 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:58.465297 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:58.465297 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:58.465297 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:58.465297 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:58.465297 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:58.465297 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 21:03:58.465297 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 21:03:58.465297 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 21:03:58.465297 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 21:03:58.466242 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:58.466242 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:58.466242 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 21:03:58.466242 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:03:58.466242 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 21:03:58.466242 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:03:58.466242 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:58.466242 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:58.466242 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:58.466242 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:58.466242 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:58.466242 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:58.466242 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:58.466242 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:58.466242 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:58.466242 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:58.466242 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:58.466242 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:58.467241 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:58.467241 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:58.467241 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 21:03:58.467241 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 21:03:58.467241 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 21:03:58.467241 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 21:03:58.467241 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 21:03:58.467241 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 21:03:58.467241 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 21:03:58.467241 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:58.467241 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 485/964] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Aug 4 21:03:58 2022] GET /identity/v3/users/cinder => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:58.472921 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5480e370-bb5f-4fd6-a30b-956bd80d09bd None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:58.581040 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5480e370-bb5f-4fd6-a30b-956bd80d09bd None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:58.616711 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a5d357-3d38-4d63-be3c-2b407aac44cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:58.617395 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a5d357-3d38-4d63-be3c-2b407aac44cd None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:58.617677 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a5d357-3d38-4d63-be3c-2b407aac44cd None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:58.617951 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a5d357-3d38-4d63-be3c-2b407aac44cd None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:58.618706 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a5d357-3d38-4d63-be3c-2b407aac44cd None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:58.619004 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a5d357-3d38-4d63-be3c-2b407aac44cd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:58.620733 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a5d357-3d38-4d63-be3c-2b407aac44cd None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:58.660208 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 480/965] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Aug 4 21:03:58 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:59.038701 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5ffea-c0da-4408-a3df-f09440abe4ec None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:59.039057 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5ffea-c0da-4408-a3df-f09440abe4ec None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:59.039356 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5ffea-c0da-4408-a3df-f09440abe4ec None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:59.040762 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 486/966] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:03:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:03:59.045891 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921c5a38-2e3a-4c32-85a1-33771a636fdd None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:59.046186 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921c5a38-2e3a-4c32-85a1-33771a636fdd None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:59.046461 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921c5a38-2e3a-4c32-85a1-33771a636fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:59.064884 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-921c5a38-2e3a-4c32-85a1-33771a636fdd None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:03:59.066369 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-921c5a38-2e3a-4c32-85a1-33771a636fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:03:59.069916 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:59.134249 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 481/967] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:03:59 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:03:59.157156 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f5ffea-c0da-4408-a3df-f09440abe4ec None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:59.271229 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f5ffea-c0da-4408-a3df-f09440abe4ec None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:59.306060 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aabcb8-3571-46c2-b196-c104bb9cb212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:59.306685 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aabcb8-3571-46c2-b196-c104bb9cb212 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:59.306916 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aabcb8-3571-46c2-b196-c104bb9cb212 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:59.307164 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aabcb8-3571-46c2-b196-c104bb9cb212 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:59.307828 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aabcb8-3571-46c2-b196-c104bb9cb212 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:59.309272 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aabcb8-3571-46c2-b196-c104bb9cb212 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:59.311222 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:59.319545 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-d6aabcb8-3571-46c2-b196-c104bb9cb212 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 21:03:59.319545 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:59.319545 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 21:03:59.319545 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 21:03:59.319545 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 21:03:59.319545 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 21:03:59.319545 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 21:03:59.319545 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:03:59.319545 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 21:03:59.319545 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:59.319545 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 21:03:59.319545 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:59.319545 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:59.319545 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:59.319545 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:59.319545 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:59.319545 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:59.324427 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:59.324427 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:59.324427 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:59.324427 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:59.324427 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:59.324427 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:59.324427 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 21:03:59.324427 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 21:03:59.324427 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 21:03:59.324427 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 21:03:59.324427 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:59.324427 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:59.324427 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:59.324427 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:59.324427 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:59.324427 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:59.325301 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:59.325301 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:59.325301 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:59.325301 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:59.325301 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:59.325301 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:59.325301 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:59.325301 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:59.325301 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:59.325301 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:59.325301 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 21:03:59.325301 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:03:59.325301 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 21:03:59.325301 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:59.325301 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 487/968] 10.222.0.25 () {56 vars in 1126 bytes} [Thu Aug 4 21:03:59 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:03:59.327789 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921c5a38-2e3a-4c32-85a1-33771a636fdd None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:59.344148 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360a09d2-dabd-4da5-9038-2e915f3cf4df None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:59.345162 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360a09d2-dabd-4da5-9038-2e915f3cf4df None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:59.345929 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360a09d2-dabd-4da5-9038-2e915f3cf4df None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:59.348365 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 488/969] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:03:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:03:59.353521 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7606e3cd-a198-4d3c-bc4a-9be42b755989 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:59.353808 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7606e3cd-a198-4d3c-bc4a-9be42b755989 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:59.354100 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7606e3cd-a198-4d3c-bc4a-9be42b755989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:59.372492 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-7606e3cd-a198-4d3c-bc4a-9be42b755989 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:03:59.373804 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-7606e3cd-a198-4d3c-bc4a-9be42b755989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:03:59.435092 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921c5a38-2e3a-4c32-85a1-33771a636fdd None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:59.453975 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 489/970] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:03:59 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:03:59.473639 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f949c6e-0603-43a6-8b8f-ba95863cfdcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:59.474263 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f949c6e-0603-43a6-8b8f-ba95863cfdcb None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:59.474546 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f949c6e-0603-43a6-8b8f-ba95863cfdcb None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:59.474822 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f949c6e-0603-43a6-8b8f-ba95863cfdcb None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:59.475532 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f949c6e-0603-43a6-8b8f-ba95863cfdcb None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:59.475840 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f949c6e-0603-43a6-8b8f-ba95863cfdcb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:59.477214 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f949c6e-0603-43a6-8b8f-ba95863cfdcb None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:59.478145 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7606e3cd-a198-4d3c-bc4a-9be42b755989 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:59.482566 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 482/971] 10.222.0.25 () {56 vars in 1122 bytes} [Thu Aug 4 21:03:59 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:59.488129 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f949c6e-0603-43a6-8b8f-ba95863cfdcb None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:59.491878 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:59.596992 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f949c6e-0603-43a6-8b8f-ba95863cfdcb None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:59.600991 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7606e3cd-a198-4d3c-bc4a-9be42b755989 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:59.630446 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:59.636655 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76bdc0f-6ea5-43d3-a17c-d13820130ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:59.637289 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76bdc0f-6ea5-43d3-a17c-d13820130ed6 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:59.637571 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76bdc0f-6ea5-43d3-a17c-d13820130ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:59.637842 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76bdc0f-6ea5-43d3-a17c-d13820130ed6 None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:59.638577 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76bdc0f-6ea5-43d3-a17c-d13820130ed6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:59.638676 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a634ffa7-aea4-4d03-a3f0-279c59ab4be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:59.639008 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76bdc0f-6ea5-43d3-a17c-d13820130ed6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:59.639304 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a634ffa7-aea4-4d03-a3f0-279c59ab4be3 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:59.639617 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a634ffa7-aea4-4d03-a3f0-279c59ab4be3 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:59.639896 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a634ffa7-aea4-4d03-a3f0-279c59ab4be3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:59.640538 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76bdc0f-6ea5-43d3-a17c-d13820130ed6 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:59.645709 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 483/972] 10.222.0.25 () {56 vars in 1122 bytes} [Thu Aug 4 21:03:59 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:59.651582 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76bdc0f-6ea5-43d3-a17c-d13820130ed6 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:59.663907 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a634ffa7-aea4-4d03-a3f0-279c59ab4be3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:03:59.669668 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-a634ffa7-aea4-4d03-a3f0-279c59ab4be3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 21:03:59.669668 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:03:59.669668 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:03:59.669668 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:03:59.669668 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:03:59.669668 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:03:59.669668 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:03:59.669668 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:03:59.669668 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:03:59.669668 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:03:59.669668 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:03:59.669668 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:03:59.669668 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:03:59.669668 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:03:59.669668 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:03:59.669668 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:03:59.670600 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:03:59.670600 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:03:59.670600 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:03:59.670600 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:03:59.670600 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:03:59.670600 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:03:59.670600 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:03:59.670600 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:03:59.670600 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:03:59.670600 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:03:59.670600 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:03:59.670600 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:03:59.670600 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:03:59.670600 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:03:59.670600 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:03:59.670600 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:03:59.672410 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:03:59.672410 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:03:59.672410 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:03:59.672410 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:03:59.672410 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:03:59.672410 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:03:59.672410 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:03:59.672410 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:03:59.672410 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:03:59.672410 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:03:59.672410 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 21:03:59.672410 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:03:59.672410 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 490/973] 10.222.0.25 () {56 vars in 1120 bytes} [Thu Aug 4 21:03:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:03:59.674373 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f905b306-269b-45d6-a5e1-62b4538009a4 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:59.674664 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f905b306-269b-45d6-a5e1-62b4538009a4 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:59.674941 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f905b306-269b-45d6-a5e1-62b4538009a4 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:59.676830 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 491/974] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:03:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:03:59.680129 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f905b306-269b-45d6-a5e1-62b4538009a4 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:59.721813 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:59.755106 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76bdc0f-6ea5-43d3-a17c-d13820130ed6 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:59.779966 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f905b306-269b-45d6-a5e1-62b4538009a4 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:59.787331 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051b9b26-50b5-4c5b-ae39-cb3e7cf3f7be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:59.787978 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051b9b26-50b5-4c5b-ae39-cb3e7cf3f7be None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:59.788258 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051b9b26-50b5-4c5b-ae39-cb3e7cf3f7be None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:59.788533 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051b9b26-50b5-4c5b-ae39-cb3e7cf3f7be None admin] PATH_INFO: `/v3/endpoints` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:59.789292 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051b9b26-50b5-4c5b-ae39-cb3e7cf3f7be None admin] RBAC: Adding query filter params (interface=admin, service_id=42b29e257c1c4000897f8dce08c20609, region_id=RegionOne) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:59.789605 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051b9b26-50b5-4c5b-ae39-cb3e7cf3f7be None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:59.790977 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051b9b26-50b5-4c5b-ae39-cb3e7cf3f7be None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:59.796173 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 484/975] 10.222.0.25 () {56 vars in 1249 bytes} [Thu Aug 4 21:03:59 2022] GET /identity/v3/endpoints?service_id=42b29e257c1c4000897f8dce08c20609&interface=admin®ion_id=RegionOne => generated 193 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:59.797297 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98efaa33-bc67-4965-8220-2a555b8a25ea None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:59.797590 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98efaa33-bc67-4965-8220-2a555b8a25ea None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:59.797867 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98efaa33-bc67-4965-8220-2a555b8a25ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:59.815663 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-98efaa33-bc67-4965-8220-2a555b8a25ea None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:03:59.816853 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-98efaa33-bc67-4965-8220-2a555b8a25ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:03:59.819458 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfec45d-bfbf-4ee9-8bf4-6614a5aa735f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:59.820085 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfec45d-bfbf-4ee9-8bf4-6614a5aa735f None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:59.820380 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfec45d-bfbf-4ee9-8bf4-6614a5aa735f None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:59.820654 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfec45d-bfbf-4ee9-8bf4-6614a5aa735f None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:59.821374 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfec45d-bfbf-4ee9-8bf4-6614a5aa735f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:03:59.821679 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfec45d-bfbf-4ee9-8bf4-6614a5aa735f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:59.825120 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfec45d-bfbf-4ee9-8bf4-6614a5aa735f None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:59.836417 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 492/976] 10.222.0.25 () {56 vars in 1115 bytes} [Thu Aug 4 21:03:59 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:03:59.837270 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfec45d-bfbf-4ee9-8bf4-6614a5aa735f None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:59.886596 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 485/977] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:03:59 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:03:59.887375 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98efaa33-bc67-4965-8220-2a555b8a25ea None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:59.892783 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:59.944399 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfec45d-bfbf-4ee9-8bf4-6614a5aa735f None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:59.947039 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:59.977138 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5b2b93-ace0-43be-9a37-9dde9d204f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:59.977756 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5b2b93-ace0-43be-9a37-9dde9d204f33 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:59.978040 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5b2b93-ace0-43be-9a37-9dde9d204f33 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:59.978317 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5b2b93-ace0-43be-9a37-9dde9d204f33 None admin] PATH_INFO: `/v3/services` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:59.978968 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5b2b93-ace0-43be-9a37-9dde9d204f33 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:59.980378 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5b2b93-ace0-43be-9a37-9dde9d204f33 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:59.986018 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98efaa33-bc67-4965-8220-2a555b8a25ea None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:59.986813 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 493/978] 10.222.0.25 () {56 vars in 1087 bytes} [Thu Aug 4 21:03:59 2022] GET /identity/v3/services => generated 1497 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:03:59.987587 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5b2b93-ace0-43be-9a37-9dde9d204f33 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:00.028631 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebbf12c-460b-4ae7-b36d-b3f1db35d9d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:00.029781 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebbf12c-460b-4ae7-b36d-b3f1db35d9d7 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:00.030199 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebbf12c-460b-4ae7-b36d-b3f1db35d9d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:00.030479 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebbf12c-460b-4ae7-b36d-b3f1db35d9d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:00.043546 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebbf12c-460b-4ae7-b36d-b3f1db35d9d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:00.046130 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebbf12c-460b-4ae7-b36d-b3f1db35d9d7 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:00.065776 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-0ebbf12c-460b-4ae7-b36d-b3f1db35d9d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:04:00.065776 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:04:00.065776 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:04:00.065776 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:04:00.065776 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:04:00.065776 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:04:00.065776 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:04:00.065776 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:04:00.065776 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:04:00.065776 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:04:00.065776 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:04:00.065776 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:04:00.065776 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:04:00.065776 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:04:00.065776 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:04:00.066781 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:04:00.066781 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:04:00.066781 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:04:00.066781 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:04:00.066781 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:04:00.066781 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:04:00.066781 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:04:00.066781 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:04:00.066781 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:04:00.066781 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:04:00.066781 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:04:00.066781 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:04:00.066781 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:04:00.066781 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:04:00.066781 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:04:00.066781 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:04:00.066781 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:04:00.067697 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:04:00.067697 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:04:00.067697 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:04:00.067697 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:04:00.071979 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 486/979] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:03:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:04:00.072821 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebbf12c-460b-4ae7-b36d-b3f1db35d9d7 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:00.117925 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5b2b93-ace0-43be-9a37-9dde9d204f33 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:00.136775 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:00.166698 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c4cc72-2160-4647-b5d6-599316216d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:00.166698 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c4cc72-2160-4647-b5d6-599316216d6d None admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:00.166698 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c4cc72-2160-4647-b5d6-599316216d6d None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:00.166698 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c4cc72-2160-4647-b5d6-599316216d6d None admin] PATH_INFO: `/v3/services` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:00.167104 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c4cc72-2160-4647-b5d6-599316216d6d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:00.168593 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c4cc72-2160-4647-b5d6-599316216d6d None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:00.187090 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebbf12c-460b-4ae7-b36d-b3f1db35d9d7 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:00.201464 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:00.221537 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4998361-4415-47b7-8660-4cfe8df93477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:00.222162 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4998361-4415-47b7-8660-4cfe8df93477 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:00.222443 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4998361-4415-47b7-8660-4cfe8df93477 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:00.222719 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4998361-4415-47b7-8660-4cfe8df93477 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:00.223440 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4998361-4415-47b7-8660-4cfe8df93477 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:04:00.223747 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4998361-4415-47b7-8660-4cfe8df93477 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:00.225127 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4998361-4415-47b7-8660-4cfe8df93477 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:00.237900 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 487/980] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:04:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:04:00.243153 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4998361-4415-47b7-8660-4cfe8df93477 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:00.247142 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 494/981] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Aug 4 21:03:59 2022] POST /identity/v3/services => generated 242 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:04:00.355588 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4998361-4415-47b7-8660-4cfe8df93477 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:00.399711 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2444a95-abcf-4089-8983-06b26845602f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:00.400290 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2444a95-abcf-4089-8983-06b26845602f None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:00.400571 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2444a95-abcf-4089-8983-06b26845602f None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:00.400844 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2444a95-abcf-4089-8983-06b26845602f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:00.409415 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2444a95-abcf-4089-8983-06b26845602f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:00.414409 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2444a95-abcf-4089-8983-06b26845602f None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:00.433481 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-e2444a95-abcf-4089-8983-06b26845602f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 21:04:00.433481 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:04:00.433481 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:04:00.433481 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:04:00.433481 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:04:00.433481 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:04:00.433481 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:04:00.433481 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:04:00.433481 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:04:00.433481 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:04:00.433481 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:04:00.433481 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:04:00.433481 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 21:04:00.433481 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 21:04:00.433481 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 21:04:00.433481 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 21:04:00.434510 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:04:00.434510 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:04:00.434510 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 21:04:00.434510 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:04:00.434510 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 21:04:00.434510 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:04:00.434510 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:04:00.434510 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:04:00.434510 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:04:00.434510 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:04:00.434510 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:04:00.434510 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:04:00.434510 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:04:00.434510 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:04:00.434510 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:04:00.434510 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:04:00.434510 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:04:00.434510 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:04:00.437336 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:04:00.437336 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:04:00.437336 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 21:04:00.437336 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 21:04:00.437336 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 21:04:00.437336 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 21:04:00.437336 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 21:04:00.437336 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 21:04:00.437336 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 21:04:00.437336 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:04:00.437336 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 488/982] 10.222.0.25 () {56 vars in 1117 bytes} [Thu Aug 4 21:04:00 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:04:00.440663 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c4cc72-2160-4647-b5d6-599316216d6d None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:00.545712 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c4cc72-2160-4647-b5d6-599316216d6d None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:00.578486 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e15d0d-cdb7-483e-b5ab-22cb115b67d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:00.579645 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e15d0d-cdb7-483e-b5ab-22cb115b67d9 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:00.579950 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e15d0d-cdb7-483e-b5ab-22cb115b67d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:00.580225 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e15d0d-cdb7-483e-b5ab-22cb115b67d9 None admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:00.581017 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e15d0d-cdb7-483e-b5ab-22cb115b67d9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:04:00.581323 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e15d0d-cdb7-483e-b5ab-22cb115b67d9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:00.582753 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e15d0d-cdb7-483e-b5ab-22cb115b67d9 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:00.605044 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 495/983] 10.222.0.25 () {56 vars in 1149 bytes} [Thu Aug 4 21:04:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:04:00.610394 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2444a95-abcf-4089-8983-06b26845602f None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:00.643063 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:00.720247 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2444a95-abcf-4089-8983-06b26845602f None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:00.757479 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d6854c-1bc3-474c-b2bb-ae2463012a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:00.758491 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d6854c-1bc3-474c-b2bb-ae2463012a63 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:00.758969 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d6854c-1bc3-474c-b2bb-ae2463012a63 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:00.759484 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d6854c-1bc3-474c-b2bb-ae2463012a63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:00.830276 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d6854c-1bc3-474c-b2bb-ae2463012a63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:00.832128 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d6854c-1bc3-474c-b2bb-ae2463012a63 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:00.865509 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:00.938417 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 489/984] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Aug 4 21:04:00 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 21:04:00.939709 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e15d0d-cdb7-483e-b5ab-22cb115b67d9 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:00.971071 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:01.043814 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e15d0d-cdb7-483e-b5ab-22cb115b67d9 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:01.086999 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35166b51-2ab3-4f90-b51f-4fd1c0117080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:01.087680 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35166b51-2ab3-4f90-b51f-4fd1c0117080 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:01.087967 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35166b51-2ab3-4f90-b51f-4fd1c0117080 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:01.088245 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35166b51-2ab3-4f90-b51f-4fd1c0117080 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:01.099303 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35166b51-2ab3-4f90-b51f-4fd1c0117080 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:01.100774 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35166b51-2ab3-4f90-b51f-4fd1c0117080 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:01.117537 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-35166b51-2ab3-4f90-b51f-4fd1c0117080 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:04:01.117537 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:04:01.117537 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:04:01.117537 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:04:01.117537 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:04:01.117537 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:04:01.117537 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:04:01.117537 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:04:01.117537 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:04:01.117537 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:04:01.117537 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:04:01.117537 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:04:01.117537 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:04:01.117537 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:04:01.117537 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:04:01.118578 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:04:01.118578 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:04:01.118578 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:04:01.118578 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:04:01.118578 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:04:01.118578 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:04:01.118578 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:04:01.118578 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:04:01.118578 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:04:01.118578 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:04:01.118578 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:04:01.118578 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:04:01.118578 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:04:01.118578 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:04:01.118578 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:04:01.118578 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:04:01.118578 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:04:01.119500 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:04:01.119500 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:04:01.119500 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:04:01.119500 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:04:01.119500 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 496/985] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:04:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:04:01.125488 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d6854c-1bc3-474c-b2bb-ae2463012a63 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:01.232543 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d6854c-1bc3-474c-b2bb-ae2463012a63 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:01.259834 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:01.265674 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11610155-c534-4b5e-9102-be843bc2bbfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:01.266288 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11610155-c534-4b5e-9102-be843bc2bbfa None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:01.266568 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11610155-c534-4b5e-9102-be843bc2bbfa None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:01.267464 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11610155-c534-4b5e-9102-be843bc2bbfa None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:01.268146 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11610155-c534-4b5e-9102-be843bc2bbfa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:04:01.268446 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11610155-c534-4b5e-9102-be843bc2bbfa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:01.269808 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11610155-c534-4b5e-9102-be843bc2bbfa None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:01.281434 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 490/986] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:04:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:04:01.287698 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35166b51-2ab3-4f90-b51f-4fd1c0117080 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:01.373775 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:01.388939 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35166b51-2ab3-4f90-b51f-4fd1c0117080 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:01.428906 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b7a3d6-ff31-417d-a0ce-8db056c12ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:01.429552 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b7a3d6-ff31-417d-a0ce-8db056c12ff4 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:01.429830 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b7a3d6-ff31-417d-a0ce-8db056c12ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:01.430099 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b7a3d6-ff31-417d-a0ce-8db056c12ff4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:01.436960 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b7a3d6-ff31-417d-a0ce-8db056c12ff4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:01.438397 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b7a3d6-ff31-417d-a0ce-8db056c12ff4 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:01.449697 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-d6b7a3d6-ff31-417d-a0ce-8db056c12ff4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:04:01.449697 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:04:01.449697 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:04:01.449697 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:04:01.449697 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:04:01.449697 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:04:01.449697 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:04:01.449697 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:04:01.449697 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:04:01.449697 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:04:01.449697 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:04:01.449697 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:04:01.449697 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 21:04:01.449697 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 21:04:01.449697 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 21:04:01.456449 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 21:04:01.456449 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:04:01.456449 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:04:01.456449 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:04:01.456449 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:04:01.456449 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:04:01.456449 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:04:01.456449 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:04:01.456449 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:04:01.456449 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:04:01.456449 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:04:01.456449 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:04:01.456449 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:04:01.456449 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:04:01.456449 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:04:01.456449 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:04:01.456449 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:04:01.457550 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 21:04:01.457550 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 21:04:01.457550 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 21:04:01.457550 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 21:04:01.457550 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 21:04:01.457550 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 21:04:01.457550 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:04:01.457550 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:04:01.457550 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 497/987] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Aug 4 21:04:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:04:01.459602 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11610155-c534-4b5e-9102-be843bc2bbfa None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:01.559267 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11610155-c534-4b5e-9102-be843bc2bbfa None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:01.593730 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73860fcb-9a87-47e4-9c31-9a689b1bedb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:01.594351 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73860fcb-9a87-47e4-9c31-9a689b1bedb9 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:01.594630 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73860fcb-9a87-47e4-9c31-9a689b1bedb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:01.594953 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73860fcb-9a87-47e4-9c31-9a689b1bedb9 None admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:01.595721 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73860fcb-9a87-47e4-9c31-9a689b1bedb9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:04:01.596028 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73860fcb-9a87-47e4-9c31-9a689b1bedb9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:01.597463 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73860fcb-9a87-47e4-9c31-9a689b1bedb9 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:01.614633 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 491/988] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Aug 4 21:04:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:04:01.615810 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b7a3d6-ff31-417d-a0ce-8db056c12ff4 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:01.721637 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b7a3d6-ff31-417d-a0ce-8db056c12ff4 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:01.755597 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71e16ef-e379-4437-808f-48a3d0807e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:01.756221 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71e16ef-e379-4437-808f-48a3d0807e00 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:01.756505 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71e16ef-e379-4437-808f-48a3d0807e00 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:01.756775 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71e16ef-e379-4437-808f-48a3d0807e00 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:01.757554 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71e16ef-e379-4437-808f-48a3d0807e00 None admin] RBAC: Adding query filter params (role.id=47f3f6a0aef34d25a88aed7f622b2d46, scope.project.id=17dc3d39e2c048ca9070588f5717a578, user.id=23ea380385f649628cc27e3603bc700c) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:04:01.757863 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71e16ef-e379-4437-808f-48a3d0807e00 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:01.759284 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71e16ef-e379-4437-808f-48a3d0807e00 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:01.771572 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 498/989] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Aug 4 21:04:01 2022] GET /identity/v3/role_assignments?user.id=23ea380385f649628cc27e3603bc700c&scope.project.id=17dc3d39e2c048ca9070588f5717a578&role.id=47f3f6a0aef34d25a88aed7f622b2d46 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:04:02.057669 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f96e78-e500-465b-841e-d66bf8ab336a None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:02.058068 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f96e78-e500-465b-841e-d66bf8ab336a None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:02.058252 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f96e78-e500-465b-841e-d66bf8ab336a None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:02.059811 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 492/990] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:04:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:04:02.065284 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bc23f4-b62c-49f6-b08c-01f75c824246 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:02.065580 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bc23f4-b62c-49f6-b08c-01f75c824246 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:02.065867 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bc23f4-b62c-49f6-b08c-01f75c824246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:02.083913 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-38bc23f4-b62c-49f6-b08c-01f75c824246 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:04:02.085109 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-38bc23f4-b62c-49f6-b08c-01f75c824246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:04:02.087310 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:02.155802 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 499/991] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:04:02 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:04:02.179950 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f96e78-e500-465b-841e-d66bf8ab336a None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:02.234550 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:02.288000 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f96e78-e500-465b-841e-d66bf8ab336a None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:02.322205 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b9e74b-03b0-4d51-a646-bee8ad4d4d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:02.322777 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b9e74b-03b0-4d51-a646-bee8ad4d4d1e None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:02.323062 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b9e74b-03b0-4d51-a646-bee8ad4d4d1e None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:02.323422 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b9e74b-03b0-4d51-a646-bee8ad4d4d1e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:02.324058 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b9e74b-03b0-4d51-a646-bee8ad4d4d1e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:02.325560 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b9e74b-03b0-4d51-a646-bee8ad4d4d1e None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:02.326528 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9358e27-e887-4493-94a4-32e7fbef2c84 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:02.326825 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9358e27-e887-4493-94a4-32e7fbef2c84 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:02.327114 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9358e27-e887-4493-94a4-32e7fbef2c84 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:02.328510 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 500/992] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:04:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:04:02.333229 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4383274-bf2e-4f4d-9c11-015a5349e9d3 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:02.333519 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4383274-bf2e-4f4d-9c11-015a5349e9d3 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:02.333800 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4383274-bf2e-4f4d-9c11-015a5349e9d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:02.335146 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-63b9e74b-03b0-4d51-a646-bee8ad4d4d1e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 21:04:02.335146 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:04:02.335146 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 21:04:02.335146 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 21:04:02.335146 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 21:04:02.335146 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 21:04:02.335146 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 21:04:02.335146 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:04:02.335146 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 21:04:02.335146 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:04:02.335146 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 21:04:02.335146 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:04:02.335146 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:04:02.335146 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:04:02.335146 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:04:02.335146 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:04:02.335146 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:04:02.336113 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:04:02.336113 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:04:02.336113 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:04:02.336113 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:04:02.336113 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:04:02.336113 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:04:02.336113 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 21:04:02.336113 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 21:04:02.336113 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 21:04:02.336113 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 21:04:02.336113 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:04:02.336113 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:04:02.336113 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:04:02.336113 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:04:02.336113 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:04:02.336113 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:04:02.336994 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:04:02.336994 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:04:02.336994 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:04:02.336994 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:04:02.336994 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:04:02.336994 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:04:02.336994 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:04:02.336994 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:04:02.336994 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:04:02.336994 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:04:02.336994 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 21:04:02.336994 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:04:02.336994 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 21:04:02.336994 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:04:02.336994 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 493/993] 10.222.0.25 () {56 vars in 1126 bytes} [Thu Aug 4 21:04:02 2022] GET /identity/v3/services/object-store => generated 93 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:04:02.342234 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b9e74b-03b0-4d51-a646-bee8ad4d4d1e None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:02.355041 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-e4383274-bf2e-4f4d-9c11-015a5349e9d3 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:04:02.356259 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-e4383274-bf2e-4f4d-9c11-015a5349e9d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:04:02.428566 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 501/994] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:04:02 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 21:04:02.450796 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4383274-bf2e-4f4d-9c11-015a5349e9d3 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:02.452591 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b9e74b-03b0-4d51-a646-bee8ad4d4d1e None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:02.468618 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:02.489895 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e524f8a-02e5-4881-8ee1-e76ad774704d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:02.490522 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e524f8a-02e5-4881-8ee1-e76ad774704d None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:02.490807 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e524f8a-02e5-4881-8ee1-e76ad774704d None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:02.491082 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e524f8a-02e5-4881-8ee1-e76ad774704d None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:02.491799 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e524f8a-02e5-4881-8ee1-e76ad774704d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:04:02.492108 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e524f8a-02e5-4881-8ee1-e76ad774704d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:02.493500 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e524f8a-02e5-4881-8ee1-e76ad774704d None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:02.503566 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 494/995] 10.222.0.25 () {56 vars in 1122 bytes} [Thu Aug 4 21:04:02 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:04:02.505237 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e524f8a-02e5-4881-8ee1-e76ad774704d None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:02.520700 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:02.555389 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4383274-bf2e-4f4d-9c11-015a5349e9d3 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:02.592027 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c012e75-6149-4d06-aeff-821e2fbd7a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:02.592604 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c012e75-6149-4d06-aeff-821e2fbd7a09 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:02.592888 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c012e75-6149-4d06-aeff-821e2fbd7a09 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:02.593167 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c012e75-6149-4d06-aeff-821e2fbd7a09 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:02.593801 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c012e75-6149-4d06-aeff-821e2fbd7a09 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:02.595765 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c012e75-6149-4d06-aeff-821e2fbd7a09 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:02.605546 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-2c012e75-6149-4d06-aeff-821e2fbd7a09 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 21:04:02.605546 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:04:02.605546 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 21:04:02.605546 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 21:04:02.605546 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 21:04:02.605546 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 21:04:02.605546 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 21:04:02.605546 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:04:02.605546 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 21:04:02.605546 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:04:02.605546 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 21:04:02.605546 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:04:02.605546 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:04:02.605546 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:04:02.605546 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:04:02.605546 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:04:02.605546 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:04:02.606595 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:04:02.606595 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:04:02.606595 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:04:02.606595 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:04:02.606595 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:04:02.606595 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:04:02.606595 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 21:04:02.606595 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 21:04:02.606595 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 21:04:02.606595 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 21:04:02.606595 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:04:02.606595 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:04:02.606595 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:04:02.606595 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:04:02.606595 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:04:02.606595 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:04:02.607499 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:04:02.607499 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:04:02.607499 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:04:02.607499 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:04:02.607499 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:04:02.607499 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:04:02.607499 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:04:02.607499 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:04:02.607499 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:04:02.607499 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:04:02.607499 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 21:04:02.607499 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:04:02.607499 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 21:04:02.607499 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:04:02.607499 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 502/996] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Aug 4 21:04:02 2022] GET /identity/v3/services/compute => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:04:02.613228 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c012e75-6149-4d06-aeff-821e2fbd7a09 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:02.616153 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e524f8a-02e5-4881-8ee1-e76ad774704d None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:02.653831 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1f297a-2661-4a84-af05-07cf51c393af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:02.654445 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1f297a-2661-4a84-af05-07cf51c393af None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:02.654729 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1f297a-2661-4a84-af05-07cf51c393af None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:02.655001 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1f297a-2661-4a84-af05-07cf51c393af None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:02.655707 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1f297a-2661-4a84-af05-07cf51c393af None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:04:02.656017 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1f297a-2661-4a84-af05-07cf51c393af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:02.658010 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1f297a-2661-4a84-af05-07cf51c393af None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:02.663446 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 495/997] 10.222.0.25 () {56 vars in 1122 bytes} [Thu Aug 4 21:04:02 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:04:02.668768 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1f297a-2661-4a84-af05-07cf51c393af None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:02.715418 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:02.720791 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c012e75-6149-4d06-aeff-821e2fbd7a09 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:02.749228 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:02.754230 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7c77e9-f0dd-4d41-9e15-948edfa2fcef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:02.754850 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c77e9-f0dd-4d41-9e15-948edfa2fcef None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:02.755183 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c77e9-f0dd-4d41-9e15-948edfa2fcef None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:02.755499 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c77e9-f0dd-4d41-9e15-948edfa2fcef None admin] PATH_INFO: `/v3/services` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:02.756171 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7c77e9-f0dd-4d41-9e15-948edfa2fcef None admin] RBAC: Adding query filter params (name=compute) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:04:02.756475 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7c77e9-f0dd-4d41-9e15-948edfa2fcef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:02.757841 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7c77e9-f0dd-4d41-9e15-948edfa2fcef None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:02.763358 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 503/998] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Aug 4 21:04:02 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:04:02.768683 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7c77e9-f0dd-4d41-9e15-948edfa2fcef None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:02.777575 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1f297a-2661-4a84-af05-07cf51c393af None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:02.812969 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32910ba7-dafa-470e-a04b-5f046a4b98a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:02.813679 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32910ba7-dafa-470e-a04b-5f046a4b98a6 None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:02.814038 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32910ba7-dafa-470e-a04b-5f046a4b98a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:02.814390 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32910ba7-dafa-470e-a04b-5f046a4b98a6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:02.815109 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32910ba7-dafa-470e-a04b-5f046a4b98a6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:02.816677 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32910ba7-dafa-470e-a04b-5f046a4b98a6 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:02.871381 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7c77e9-f0dd-4d41-9e15-948edfa2fcef None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:02.891237 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 496/999] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:04:02 2022] POST /identity/v3/endpoints => generated 325 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:04:02.916983 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd01e04-8c05-4f83-9321-f626eafca74c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:02.917638 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd01e04-8c05-4f83-9321-f626eafca74c None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:02.917920 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd01e04-8c05-4f83-9321-f626eafca74c None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:02.918270 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd01e04-8c05-4f83-9321-f626eafca74c None admin] PATH_INFO: `/v3/services` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:02.918984 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd01e04-8c05-4f83-9321-f626eafca74c None admin] RBAC: Adding query filter params (type=compute) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:04:02.919289 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd01e04-8c05-4f83-9321-f626eafca74c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:02.920759 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd01e04-8c05-4f83-9321-f626eafca74c None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:02.926825 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 504/1000] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Aug 4 21:04:02 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:04:02.932457 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32910ba7-dafa-470e-a04b-5f046a4b98a6 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:03.044876 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32910ba7-dafa-470e-a04b-5f046a4b98a6 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:03.046590 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:03.081936 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44497791-3717-4bab-8544-99c97a77ac5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:03.082575 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44497791-3717-4bab-8544-99c97a77ac5d None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:03.082863 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44497791-3717-4bab-8544-99c97a77ac5d None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:03.083141 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44497791-3717-4bab-8544-99c97a77ac5d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:03.083953 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44497791-3717-4bab-8544-99c97a77ac5d None admin] RBAC: Adding query filter params (interface=public, service_id=f4e0bd494c69482b80ac70de587afa24, region_id=RegionOne) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:04:03.084266 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44497791-3717-4bab-8544-99c97a77ac5d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:03.085647 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44497791-3717-4bab-8544-99c97a77ac5d None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:03.090786 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 497/1001] 10.222.0.25 () {56 vars in 1251 bytes} [Thu Aug 4 21:04:02 2022] GET /identity/v3/endpoints?service_id=f4e0bd494c69482b80ac70de587afa24&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:04:03.096486 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd01e04-8c05-4f83-9321-f626eafca74c None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:03.156489 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:03.200411 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd01e04-8c05-4f83-9321-f626eafca74c None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:03.240467 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc58052-27a0-4bbe-8e9b-754b3aa6fe56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:03.241098 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc58052-27a0-4bbe-8e9b-754b3aa6fe56 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:03.241378 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc58052-27a0-4bbe-8e9b-754b3aa6fe56 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:03.241650 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc58052-27a0-4bbe-8e9b-754b3aa6fe56 None admin] PATH_INFO: `/v3/services` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:03.242303 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc58052-27a0-4bbe-8e9b-754b3aa6fe56 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:03.243695 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc58052-27a0-4bbe-8e9b-754b3aa6fe56 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:03.255404 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 505/1002] 10.222.0.25 () {56 vars in 1087 bytes} [Thu Aug 4 21:04:03 2022] GET /identity/v3/services => generated 1727 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:04:03.834156 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ecdc6e-c73a-4e39-8b4a-86f335190c65 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:03.834673 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ecdc6e-c73a-4e39-8b4a-86f335190c65 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:03.834750 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ecdc6e-c73a-4e39-8b4a-86f335190c65 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:03.836260 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 498/1003] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:04:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 21:04:03.841089 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df71e11f-94fd-4063-8686-7785b24a1084 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:03.841465 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df71e11f-94fd-4063-8686-7785b24a1084 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:03.841642 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df71e11f-94fd-4063-8686-7785b24a1084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:03.860098 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-df71e11f-94fd-4063-8686-7785b24a1084 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:04:03.861860 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-df71e11f-94fd-4063-8686-7785b24a1084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:04:03.929636 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 506/1004] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:04:03 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:04:03.950763 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ecdc6e-c73a-4e39-8b4a-86f335190c65 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:04.052574 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ecdc6e-c73a-4e39-8b4a-86f335190c65 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:04.086191 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4978aff7-94db-4f4d-ac4c-b2fd0ac7572e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:04.086751 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4978aff7-94db-4f4d-ac4c-b2fd0ac7572e None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:04.087034 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4978aff7-94db-4f4d-ac4c-b2fd0ac7572e None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:04.087350 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4978aff7-94db-4f4d-ac4c-b2fd0ac7572e None admin] PATH_INFO: `/v3/services/image` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:04.087980 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4978aff7-94db-4f4d-ac4c-b2fd0ac7572e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:04.089352 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4978aff7-94db-4f4d-ac4c-b2fd0ac7572e None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:04.098658 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-4978aff7-94db-4f4d-ac4c-b2fd0ac7572e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 21:04:04.098658 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:04:04.098658 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 21:04:04.098658 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 21:04:04.098658 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 21:04:04.098658 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 21:04:04.098658 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 21:04:04.098658 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:04:04.098658 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 21:04:04.098658 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:04:04.098658 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 21:04:04.098658 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:04:04.098658 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:04:04.098658 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:04:04.098658 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:04:04.098658 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:04:04.098658 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:04:04.099936 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:04:04.099936 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:04:04.099936 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:04:04.099936 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:04:04.099936 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:04:04.099936 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:04:04.099936 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 21:04:04.099936 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 21:04:04.099936 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 21:04:04.099936 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 21:04:04.099936 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:04:04.099936 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:04:04.099936 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:04:04.099936 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:04:04.099936 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:04:04.099936 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:04:04.101172 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:04:04.101172 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:04:04.101172 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:04:04.101172 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:04:04.101172 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:04:04.101172 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:04:04.101172 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:04:04.101172 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:04:04.101172 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:04:04.101172 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:04:04.101172 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 21:04:04.101172 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:04:04.101172 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 21:04:04.101172 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:04:04.103685 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 499/1005] 10.222.0.25 () {56 vars in 1105 bytes} [Thu Aug 4 21:04:03 2022] GET /identity/v3/services/image => generated 86 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:04:04.105621 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df71e11f-94fd-4063-8686-7785b24a1084 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:04.155848 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:04.214164 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df71e11f-94fd-4063-8686-7785b24a1084 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:04.253107 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd490d3-4a59-4a17-9d27-f6e9f93a6bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:04.253733 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd490d3-4a59-4a17-9d27-f6e9f93a6bac None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:04.254014 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd490d3-4a59-4a17-9d27-f6e9f93a6bac None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:04.254284 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd490d3-4a59-4a17-9d27-f6e9f93a6bac None admin] PATH_INFO: `/v3/services` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:04.254960 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd490d3-4a59-4a17-9d27-f6e9f93a6bac None admin] RBAC: Adding query filter params (name=image) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:04:04.255263 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd490d3-4a59-4a17-9d27-f6e9f93a6bac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:04.256685 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd490d3-4a59-4a17-9d27-f6e9f93a6bac None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:04.261582 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 507/1006] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:04:04 2022] GET /identity/v3/services?name=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:04:04.267548 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4978aff7-94db-4f4d-ac4c-b2fd0ac7572e None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:04.298690 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:04.370270 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4978aff7-94db-4f4d-ac4c-b2fd0ac7572e None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:04.404506 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aac316-be45-4ab8-83c4-d9a4cd5a1ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:04.405267 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aac316-be45-4ab8-83c4-d9a4cd5a1ec4 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:04.405649 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aac316-be45-4ab8-83c4-d9a4cd5a1ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:04.406021 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aac316-be45-4ab8-83c4-d9a4cd5a1ec4 None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:04.406792 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aac316-be45-4ab8-83c4-d9a4cd5a1ec4 None admin] RBAC: Adding query filter params (type=image) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:04:04.407250 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aac316-be45-4ab8-83c4-d9a4cd5a1ec4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:04.409487 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aac316-be45-4ab8-83c4-d9a4cd5a1ec4 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:04.414263 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 500/1007] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:04:04 2022] GET /identity/v3/services?type=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:04:04.968875 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6586768-6806-4100-99e0-237344e4f1dd None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:04.969425 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6586768-6806-4100-99e0-237344e4f1dd None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:04.969425 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6586768-6806-4100-99e0-237344e4f1dd None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:04.971488 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 508/1008] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:04:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:04:04.976480 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d911224b-8a48-4ee4-9e52-5d4e4d345de0 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:04.976860 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d911224b-8a48-4ee4-9e52-5d4e4d345de0 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:04.977043 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d911224b-8a48-4ee4-9e52-5d4e4d345de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:04.995199 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-d911224b-8a48-4ee4-9e52-5d4e4d345de0 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:04:04.996417 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-d911224b-8a48-4ee4-9e52-5d4e4d345de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:04:05.066976 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 501/1009] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:04:04 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:04:05.089937 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6586768-6806-4100-99e0-237344e4f1dd None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:05.192477 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6586768-6806-4100-99e0-237344e4f1dd None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:05.206731 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:05.229908 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ef6003-0710-4b8b-a199-c5ce31ef33f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:05.230551 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ef6003-0710-4b8b-a199-c5ce31ef33f4 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:05.230831 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ef6003-0710-4b8b-a199-c5ce31ef33f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:05.231103 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ef6003-0710-4b8b-a199-c5ce31ef33f4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:05.232344 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ef6003-0710-4b8b-a199-c5ce31ef33f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:05.233762 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ef6003-0710-4b8b-a199-c5ce31ef33f4 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:05.236063 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 509/1010] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:04:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:04:05.241831 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d911224b-8a48-4ee4-9e52-5d4e4d345de0 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:05.260037 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7f5452-c199-4137-8174-945f24bc4001 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:05.260659 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7f5452-c199-4137-8174-945f24bc4001 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:05.261389 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7f5452-c199-4137-8174-945f24bc4001 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:05.263184 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 510/1011] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:04:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 21:04:05.267749 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0c94a8-0fbc-4efe-b3fb-a61e0977a0dd None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:05.268044 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:05.268164 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0c94a8-0fbc-4efe-b3fb-a61e0977a0dd None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:05.268432 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0c94a8-0fbc-4efe-b3fb-a61e0977a0dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:05.287573 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-fe0c94a8-0fbc-4efe-b3fb-a61e0977a0dd None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:04:05.288917 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-fe0c94a8-0fbc-4efe-b3fb-a61e0977a0dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:04:05.348231 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d911224b-8a48-4ee4-9e52-5d4e4d345de0 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:05.363424 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 511/1012] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:04:05 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:04:05.382111 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0c94a8-0fbc-4efe-b3fb-a61e0977a0dd None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:05.387339 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8305fe1-0b26-4e0c-b5c5-cf40770c8e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:05.387971 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8305fe1-0b26-4e0c-b5c5-cf40770c8e5a None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:05.388252 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8305fe1-0b26-4e0c-b5c5-cf40770c8e5a None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:05.388524 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8305fe1-0b26-4e0c-b5c5-cf40770c8e5a None admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:05.389207 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8305fe1-0b26-4e0c-b5c5-cf40770c8e5a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:05.391057 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8305fe1-0b26-4e0c-b5c5-cf40770c8e5a None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:05.488630 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 502/1013] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Aug 4 21:04:05 2022] POST /identity/v3/projects => generated 311 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:04:05.488815 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0c94a8-0fbc-4efe-b3fb-a61e0977a0dd None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:05.504857 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:05.528788 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c22fcf-fba7-4986-8d19-b77f6f791112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:05.529523 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c22fcf-fba7-4986-8d19-b77f6f791112 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:05.529836 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c22fcf-fba7-4986-8d19-b77f6f791112 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:05.530116 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c22fcf-fba7-4986-8d19-b77f6f791112 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:05.530763 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c22fcf-fba7-4986-8d19-b77f6f791112 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:05.532729 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c22fcf-fba7-4986-8d19-b77f6f791112 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:05.542951 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-89c22fcf-fba7-4986-8d19-b77f6f791112 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 21:04:05.542951 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:04:05.542951 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 21:04:05.542951 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 21:04:05.542951 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 21:04:05.542951 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 21:04:05.542951 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 21:04:05.542951 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:04:05.542951 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 21:04:05.542951 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:04:05.542951 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 21:04:05.542951 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:04:05.542951 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:04:05.542951 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:04:05.542951 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:04:05.542951 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:04:05.542951 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:04:05.544052 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:04:05.544052 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:04:05.544052 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:04:05.544052 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:04:05.544052 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:04:05.544052 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:04:05.544052 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 21:04:05.544052 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 21:04:05.544052 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 21:04:05.544052 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 21:04:05.544052 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:04:05.544052 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:04:05.544052 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:04:05.544052 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:04:05.544052 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:04:05.544052 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:04:05.545106 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:04:05.545106 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:04:05.545106 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:04:05.545106 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:04:05.545106 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:04:05.545106 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:04:05.545106 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:04:05.545106 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:04:05.545106 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:04:05.545106 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:04:05.545106 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 21:04:05.545106 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:04:05.545106 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 21:04:05.545106 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:04:05.545106 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 512/1014] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Aug 4 21:04:05 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:04:05.550991 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8305fe1-0b26-4e0c-b5c5-cf40770c8e5a None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:05.634120 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:05.663729 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8305fe1-0b26-4e0c-b5c5-cf40770c8e5a None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:05.712228 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617a17ae-4ee6-4514-999e-471038b078a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:05.713193 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617a17ae-4ee6-4514-999e-471038b078a7 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:05.713635 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617a17ae-4ee6-4514-999e-471038b078a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:05.714088 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617a17ae-4ee6-4514-999e-471038b078a7 None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:05.715249 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617a17ae-4ee6-4514-999e-471038b078a7 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:04:05.715951 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617a17ae-4ee6-4514-999e-471038b078a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:05.717215 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617a17ae-4ee6-4514-999e-471038b078a7 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:05.722908 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 503/1015] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Aug 4 21:04:05 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:04:05.728858 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c22fcf-fba7-4986-8d19-b77f6f791112 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:05.842688 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c22fcf-fba7-4986-8d19-b77f6f791112 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:05.876721 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73276f33-f252-4759-94ad-0ec869f11c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:05.877351 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73276f33-f252-4759-94ad-0ec869f11c55 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:05.877631 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73276f33-f252-4759-94ad-0ec869f11c55 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:05.877900 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73276f33-f252-4759-94ad-0ec869f11c55 None admin] PATH_INFO: `/v3/services` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:05.878574 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73276f33-f252-4759-94ad-0ec869f11c55 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:04:05.878873 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73276f33-f252-4759-94ad-0ec869f11c55 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:05.880260 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73276f33-f252-4759-94ad-0ec869f11c55 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:05.885254 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 513/1016] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Aug 4 21:04:05 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:04:05.890676 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617a17ae-4ee6-4514-999e-471038b078a7 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:05.991941 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617a17ae-4ee6-4514-999e-471038b078a7 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:06.028140 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e20327-f065-4bde-8f52-ed3076a6932d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:06.028760 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e20327-f065-4bde-8f52-ed3076a6932d None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:06.029040 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e20327-f065-4bde-8f52-ed3076a6932d None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:06.029314 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e20327-f065-4bde-8f52-ed3076a6932d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:06.030031 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e20327-f065-4bde-8f52-ed3076a6932d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:06.031530 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e20327-f065-4bde-8f52-ed3076a6932d None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:06.041979 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:06.155156 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 504/1017] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:04:05 2022] POST /identity/v3/endpoints => generated 334 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:04:06.420716 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8e3540-5a9a-400e-9f34-79f1788d1065 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:06.421243 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8e3540-5a9a-400e-9f34-79f1788d1065 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:06.421243 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8e3540-5a9a-400e-9f34-79f1788d1065 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:06.422613 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 514/1018] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:04:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:04:06.428337 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7224691c-ad20-437d-80e9-257040a8c686 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:06.428629 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7224691c-ad20-437d-80e9-257040a8c686 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:06.428909 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7224691c-ad20-437d-80e9-257040a8c686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:06.447353 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-7224691c-ad20-437d-80e9-257040a8c686 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:04:06.448541 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-7224691c-ad20-437d-80e9-257040a8c686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:04:06.522027 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 505/1019] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:04:06 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:04:06.543732 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8e3540-5a9a-400e-9f34-79f1788d1065 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:06.588195 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:06.648361 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8e3540-5a9a-400e-9f34-79f1788d1065 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:06.684420 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e86989e-d4ab-49ee-9d87-19f7541107ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:06.684960 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86989e-d4ab-49ee-9d87-19f7541107ba None admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:06.685242 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86989e-d4ab-49ee-9d87-19f7541107ba None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:06.685512 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86989e-d4ab-49ee-9d87-19f7541107ba None admin] PATH_INFO: `/v3/services` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:06.686220 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e86989e-d4ab-49ee-9d87-19f7541107ba None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:06.687609 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e86989e-d4ab-49ee-9d87-19f7541107ba None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:06.789801 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 515/1020] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Aug 4 21:04:06 2022] POST /identity/v3/services => generated 242 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:04:07.568394 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106d07ff-4e79-45f2-86ec-64672cf16e19 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:07.568886 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106d07ff-4e79-45f2-86ec-64672cf16e19 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:07.568886 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106d07ff-4e79-45f2-86ec-64672cf16e19 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:07.570379 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 506/1021] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:04:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:04:07.575472 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6cfab8-aaa5-4932-9b02-45f570715337 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:07.575868 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6cfab8-aaa5-4932-9b02-45f570715337 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:07.576077 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6cfab8-aaa5-4932-9b02-45f570715337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:07.594128 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-5e6cfab8-aaa5-4932-9b02-45f570715337 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:04:07.595862 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-5e6cfab8-aaa5-4932-9b02-45f570715337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:04:07.669596 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 516/1022] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:04:07 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:04:07.698515 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106d07ff-4e79-45f2-86ec-64672cf16e19 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:07.803012 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106d07ff-4e79-45f2-86ec-64672cf16e19 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:07.825685 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:07.837448 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169dd125-5333-4a40-98f0-2c31b98bfcb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:07.838097 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169dd125-5333-4a40-98f0-2c31b98bfcb1 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:07.838380 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169dd125-5333-4a40-98f0-2c31b98bfcb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:07.840851 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169dd125-5333-4a40-98f0-2c31b98bfcb1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:07.840851 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169dd125-5333-4a40-98f0-2c31b98bfcb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:07.841257 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169dd125-5333-4a40-98f0-2c31b98bfcb1 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:07.843020 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 507/1023] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:04:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:04:07.849265 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6cfab8-aaa5-4932-9b02-45f570715337 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:07.882693 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:07.960286 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6cfab8-aaa5-4932-9b02-45f570715337 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:07.994871 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79af691e-4e3e-4136-a995-3abc81545da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:07.995526 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79af691e-4e3e-4136-a995-3abc81545da7 None admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:07.995811 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79af691e-4e3e-4136-a995-3abc81545da7 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:07.996085 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79af691e-4e3e-4136-a995-3abc81545da7 None admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:07.996784 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79af691e-4e3e-4136-a995-3abc81545da7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:07.998259 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79af691e-4e3e-4136-a995-3abc81545da7 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:08.097517 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 517/1024] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Aug 4 21:04:07 2022] POST /identity/v3/users => generated 285 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:04:08.278816 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4cac2b-7e2a-42ea-8c9c-35e0a7195cfc None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:08.279167 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4cac2b-7e2a-42ea-8c9c-35e0a7195cfc None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:08.279428 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4cac2b-7e2a-42ea-8c9c-35e0a7195cfc None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:08.282896 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 508/1025] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:04:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:04:08.288602 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b71f327-4b8c-43e1-b3b1-24db92e90269 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:08.288892 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b71f327-4b8c-43e1-b3b1-24db92e90269 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:08.289171 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b71f327-4b8c-43e1-b3b1-24db92e90269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:08.307855 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-1b71f327-4b8c-43e1-b3b1-24db92e90269 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:04:08.309349 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-1b71f327-4b8c-43e1-b3b1-24db92e90269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:04:08.380037 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 518/1026] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:04:08 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:04:08.403253 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4cac2b-7e2a-42ea-8c9c-35e0a7195cfc None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:08.507115 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4cac2b-7e2a-42ea-8c9c-35e0a7195cfc None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:08.541274 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13ac57b-3f6a-42db-b245-d85794f49335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:08.541914 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13ac57b-3f6a-42db-b245-d85794f49335 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:08.542196 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13ac57b-3f6a-42db-b245-d85794f49335 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:08.542610 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13ac57b-3f6a-42db-b245-d85794f49335 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:08.552562 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13ac57b-3f6a-42db-b245-d85794f49335 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:04:08.557418 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-d13ac57b-3f6a-42db-b245-d85794f49335 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 21:04:08.557418 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:04:08.557418 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:04:08.557418 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:04:08.557418 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:04:08.557418 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:04:08.557418 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:04:08.557418 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:04:08.557418 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:04:08.557418 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:04:08.557418 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:04:08.557418 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:04:08.557418 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:04:08.557418 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:04:08.557418 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:04:08.557418 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:04:08.558341 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:04:08.558341 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:04:08.558341 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:04:08.558341 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:04:08.558341 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:04:08.558341 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:04:08.558341 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:04:08.558341 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:04:08.558341 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:04:08.558341 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:04:08.558341 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:04:08.558341 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:04:08.558341 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:04:08.558341 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:04:08.558341 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:04:08.558341 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:04:08.559183 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:04:08.559183 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:04:08.559183 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:04:08.559183 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:04:08.559183 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:04:08.559183 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:04:08.559183 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:04:08.559183 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:04:08.559183 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:04:08.559183 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:04:08.559183 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 21:04:08.559183 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:04:08.559183 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 509/1027] 10.222.0.25 () {56 vars in 1120 bytes} [Thu Aug 4 21:04:08 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:04:08.567290 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b71f327-4b8c-43e1-b3b1-24db92e90269 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:08.582797 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:08.668346 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b71f327-4b8c-43e1-b3b1-24db92e90269 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:08.702670 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2268492e-9503-4d1a-99df-01577b55bd12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:08.703335 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2268492e-9503-4d1a-99df-01577b55bd12 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:08.703688 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2268492e-9503-4d1a-99df-01577b55bd12 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:08.704015 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2268492e-9503-4d1a-99df-01577b55bd12 None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:08.704714 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2268492e-9503-4d1a-99df-01577b55bd12 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:04:08.705046 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2268492e-9503-4d1a-99df-01577b55bd12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:08.706414 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2268492e-9503-4d1a-99df-01577b55bd12 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:08.719214 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 519/1028] 10.222.0.25 () {56 vars in 1115 bytes} [Thu Aug 4 21:04:08 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:04:08.720847 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13ac57b-3f6a-42db-b245-d85794f49335 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:08.782415 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:08.832665 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13ac57b-3f6a-42db-b245-d85794f49335 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:08.847241 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4cb5ed-7a73-4fed-9ec3-ac7ff2eac999 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:08.847579 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4cb5ed-7a73-4fed-9ec3-ac7ff2eac999 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:08.847842 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4cb5ed-7a73-4fed-9ec3-ac7ff2eac999 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:08.849327 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 520/1029] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:04:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 21:04:08.853986 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b54f938-b324-415e-99cc-98ea305f9510 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:08.854276 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b54f938-b324-415e-99cc-98ea305f9510 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:08.854556 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b54f938-b324-415e-99cc-98ea305f9510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:08.869600 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b9e14a-3b3e-42a7-91db-3321d50f1d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:08.870244 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b9e14a-3b3e-42a7-91db-3321d50f1d61 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:08.870533 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b9e14a-3b3e-42a7-91db-3321d50f1d61 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:08.870807 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b9e14a-3b3e-42a7-91db-3321d50f1d61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:08.872680 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-2b54f938-b324-415e-99cc-98ea305f9510 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:04:08.873865 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-2b54f938-b324-415e-99cc-98ea305f9510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:04:08.882052 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b9e14a-3b3e-42a7-91db-3321d50f1d61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:08.883548 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b9e14a-3b3e-42a7-91db-3321d50f1d61 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:08.898039 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-a4b9e14a-3b3e-42a7-91db-3321d50f1d61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:04:08.898039 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:04:08.898039 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:04:08.898039 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:04:08.898039 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:04:08.898039 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:04:08.898039 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:04:08.898039 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:04:08.898039 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:04:08.898039 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:04:08.898039 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:04:08.898039 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:04:08.898039 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:04:08.898039 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:04:08.898039 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:04:08.899151 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:04:08.899151 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:04:08.899151 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:04:08.899151 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:04:08.899151 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:04:08.899151 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:04:08.899151 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:04:08.899151 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:04:08.899151 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:04:08.899151 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:04:08.899151 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:04:08.899151 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:04:08.899151 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:04:08.899151 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:04:08.899151 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:04:08.899151 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:04:08.899151 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:04:08.900140 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:04:08.900140 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:04:08.900140 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:04:08.900140 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:04:08.900140 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 510/1030] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:04:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:04:08.904695 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b9e14a-3b3e-42a7-91db-3321d50f1d61 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:08.948124 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 521/1031] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:04:08 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 21:04:08.969874 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b54f938-b324-415e-99cc-98ea305f9510 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:08.972767 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:09.010828 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b9e14a-3b3e-42a7-91db-3321d50f1d61 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:09.013059 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:09.048586 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bad121-2f00-4d08-93bc-a3ff763d1f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:09.049213 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bad121-2f00-4d08-93bc-a3ff763d1f52 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:09.049498 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bad121-2f00-4d08-93bc-a3ff763d1f52 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:09.049762 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bad121-2f00-4d08-93bc-a3ff763d1f52 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:09.050456 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bad121-2f00-4d08-93bc-a3ff763d1f52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:04:09.050764 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bad121-2f00-4d08-93bc-a3ff763d1f52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:09.052173 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bad121-2f00-4d08-93bc-a3ff763d1f52 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:09.065558 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 511/1032] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:04:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:04:09.071271 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bad121-2f00-4d08-93bc-a3ff763d1f52 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:09.073584 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b54f938-b324-415e-99cc-98ea305f9510 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:09.112126 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154d3721-7773-40a8-9622-631f9553e061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:09.112772 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d3721-7773-40a8-9622-631f9553e061 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:09.113054 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d3721-7773-40a8-9622-631f9553e061 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:09.113328 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d3721-7773-40a8-9622-631f9553e061 None admin] PATH_INFO: `/v3/services/image` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:09.114011 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154d3721-7773-40a8-9622-631f9553e061 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:09.115440 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154d3721-7773-40a8-9622-631f9553e061 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:09.124337 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-154d3721-7773-40a8-9622-631f9553e061 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 21:04:09.124337 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:04:09.124337 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 21:04:09.124337 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 21:04:09.124337 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 21:04:09.124337 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 21:04:09.124337 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 21:04:09.124337 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:04:09.124337 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 21:04:09.124337 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:04:09.124337 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 21:04:09.124337 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:04:09.124337 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:04:09.124337 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:04:09.124337 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:04:09.124337 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:04:09.124337 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:04:09.125393 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:04:09.125393 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:04:09.125393 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:04:09.125393 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:04:09.125393 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:04:09.125393 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:04:09.125393 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 21:04:09.125393 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 21:04:09.125393 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 21:04:09.125393 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 21:04:09.125393 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:04:09.125393 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:04:09.125393 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:04:09.125393 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:04:09.125393 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:04:09.125393 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:04:09.126264 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:04:09.126264 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:04:09.126264 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:04:09.126264 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:04:09.126264 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:04:09.126264 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:04:09.126264 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:04:09.126264 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:04:09.126264 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:04:09.126264 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:04:09.126264 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 21:04:09.126264 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:04:09.126264 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 21:04:09.126264 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:04:09.130534 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 522/1033] 10.222.0.25 () {56 vars in 1105 bytes} [Thu Aug 4 21:04:08 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:04:09.131381 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154d3721-7773-40a8-9622-631f9553e061 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:09.178034 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bad121-2f00-4d08-93bc-a3ff763d1f52 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:09.201356 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:09.215044 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a48bdc-587b-483f-8241-ed7ac06cde85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:09.215724 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a48bdc-587b-483f-8241-ed7ac06cde85 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:09.216010 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a48bdc-587b-483f-8241-ed7ac06cde85 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:09.216289 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a48bdc-587b-483f-8241-ed7ac06cde85 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:09.225262 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a48bdc-587b-483f-8241-ed7ac06cde85 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:09.226767 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a48bdc-587b-483f-8241-ed7ac06cde85 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:09.238695 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-59a48bdc-587b-483f-8241-ed7ac06cde85 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 21:04:09.238695 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:04:09.238695 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:04:09.238695 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:04:09.238695 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:04:09.238695 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:04:09.238695 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:04:09.238695 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:04:09.238695 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:04:09.238695 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:04:09.238695 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:04:09.238695 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:04:09.238695 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 21:04:09.238695 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 21:04:09.238695 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 21:04:09.238695 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 21:04:09.239731 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:04:09.239731 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:04:09.239731 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 21:04:09.239731 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:04:09.239731 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 21:04:09.239731 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:04:09.239731 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:04:09.239731 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:04:09.239731 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:04:09.239731 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:04:09.239731 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:04:09.239731 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:04:09.239731 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:04:09.239731 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:04:09.239731 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:04:09.239731 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:04:09.239731 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:04:09.239731 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:04:09.240732 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:04:09.240732 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:04:09.240732 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 21:04:09.240732 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 21:04:09.240732 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 21:04:09.240732 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 21:04:09.240732 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 21:04:09.240732 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 21:04:09.240732 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 21:04:09.240732 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:04:09.244678 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 512/1034] 10.222.0.25 () {56 vars in 1093 bytes} [Thu Aug 4 21:04:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:04:09.245501 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a48bdc-587b-483f-8241-ed7ac06cde85 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:09.246995 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154d3721-7773-40a8-9622-631f9553e061 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:09.273055 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:09.285440 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152adab4-5954-44c1-8812-c76cb805eec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:09.286057 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152adab4-5954-44c1-8812-c76cb805eec9 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:09.286342 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152adab4-5954-44c1-8812-c76cb805eec9 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:09.286618 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152adab4-5954-44c1-8812-c76cb805eec9 None admin] PATH_INFO: `/v3/services` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:09.287299 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152adab4-5954-44c1-8812-c76cb805eec9 None admin] RBAC: Adding query filter params (name=image) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:04:09.287630 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152adab4-5954-44c1-8812-c76cb805eec9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:09.289005 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152adab4-5954-44c1-8812-c76cb805eec9 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:09.294238 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 523/1035] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:04:09 2022] GET /identity/v3/services?name=image => generated 122 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:04:09.300376 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152adab4-5954-44c1-8812-c76cb805eec9 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:09.357513 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a48bdc-587b-483f-8241-ed7ac06cde85 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:09.392576 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8940af41-768a-42a7-b68a-5b47fa8323c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:09.393204 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8940af41-768a-42a7-b68a-5b47fa8323c8 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:09.393489 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8940af41-768a-42a7-b68a-5b47fa8323c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:09.393767 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8940af41-768a-42a7-b68a-5b47fa8323c8 None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:09.394677 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8940af41-768a-42a7-b68a-5b47fa8323c8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:04:09.394988 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8940af41-768a-42a7-b68a-5b47fa8323c8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:09.396456 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8940af41-768a-42a7-b68a-5b47fa8323c8 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:09.406659 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152adab4-5954-44c1-8812-c76cb805eec9 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:09.414460 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 513/1036] 10.222.0.25 () {56 vars in 1133 bytes} [Thu Aug 4 21:04:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:04:09.420111 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8940af41-768a-42a7-b68a-5b47fa8323c8 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:09.437563 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:09.443803 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428a54fa-134d-4a46-a860-66d4848a844e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:09.444432 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428a54fa-134d-4a46-a860-66d4848a844e None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:09.444724 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428a54fa-134d-4a46-a860-66d4848a844e None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:09.445007 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428a54fa-134d-4a46-a860-66d4848a844e None admin] PATH_INFO: `/v3/services` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:09.445695 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428a54fa-134d-4a46-a860-66d4848a844e None admin] RBAC: Adding query filter params (type=image) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:04:09.446003 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428a54fa-134d-4a46-a860-66d4848a844e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:09.447509 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428a54fa-134d-4a46-a860-66d4848a844e None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:09.455413 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 524/1037] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:04:09 2022] GET /identity/v3/services?type=image => generated 350 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:04:09.458482 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428a54fa-134d-4a46-a860-66d4848a844e None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:09.507815 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:09.529611 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8940af41-768a-42a7-b68a-5b47fa8323c8 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:09.567753 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a5af02-d060-41cf-8121-2b2ecc7c6a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:09.568700 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a5af02-d060-41cf-8121-2b2ecc7c6a7a None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:09.568994 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a5af02-d060-41cf-8121-2b2ecc7c6a7a None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:09.569275 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a5af02-d060-41cf-8121-2b2ecc7c6a7a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:09.571436 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428a54fa-134d-4a46-a860-66d4848a844e None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:09.610540 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b4e18d-fa5f-470c-ad0e-133e7635724a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:09.611196 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b4e18d-fa5f-470c-ad0e-133e7635724a None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:09.611524 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b4e18d-fa5f-470c-ad0e-133e7635724a None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:09.611803 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b4e18d-fa5f-470c-ad0e-133e7635724a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:09.613032 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b4e18d-fa5f-470c-ad0e-133e7635724a None admin] RBAC: Adding query filter params (interface=public, service_id=3e7ff50d6f8c47ca9ad89dabba08e852, region_id=RegionOne) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:04:09.613347 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b4e18d-fa5f-470c-ad0e-133e7635724a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:09.615166 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b4e18d-fa5f-470c-ad0e-133e7635724a None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:09.626540 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 525/1038] 10.222.0.25 () {56 vars in 1251 bytes} [Thu Aug 4 21:04:09 2022] GET /identity/v3/endpoints?service_id=3e7ff50d6f8c47ca9ad89dabba08e852&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:04:09.627587 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b4e18d-fa5f-470c-ad0e-133e7635724a None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:09.642871 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a5af02-d060-41cf-8121-2b2ecc7c6a7a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:09.644505 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a5af02-d060-41cf-8121-2b2ecc7c6a7a None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:09.694542 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:09.736762 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:09.753121 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b4e18d-fa5f-470c-ad0e-133e7635724a None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:09.762611 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 514/1039] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Aug 4 21:04:09 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 21:04:09.768820 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a5af02-d060-41cf-8121-2b2ecc7c6a7a None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:09.793955 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d276b922-f7d8-4af5-813a-b5ab2f831685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:09.794579 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d276b922-f7d8-4af5-813a-b5ab2f831685 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:09.794864 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d276b922-f7d8-4af5-813a-b5ab2f831685 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:09.795135 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d276b922-f7d8-4af5-813a-b5ab2f831685 None admin] PATH_INFO: `/v3/services` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:09.795819 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d276b922-f7d8-4af5-813a-b5ab2f831685 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:09.797228 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d276b922-f7d8-4af5-813a-b5ab2f831685 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:09.815353 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 526/1040] 10.222.0.25 () {56 vars in 1087 bytes} [Thu Aug 4 21:04:09 2022] GET /identity/v3/services => generated 1957 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:04:09.889304 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a5af02-d060-41cf-8121-2b2ecc7c6a7a None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:09.926333 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c6646a-36b1-4c43-ab8f-5ddac012d7ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:09.926997 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c6646a-36b1-4c43-ab8f-5ddac012d7ca None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:09.927284 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c6646a-36b1-4c43-ab8f-5ddac012d7ca None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:09.927589 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c6646a-36b1-4c43-ab8f-5ddac012d7ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:09.939428 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c6646a-36b1-4c43-ab8f-5ddac012d7ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:09.940939 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c6646a-36b1-4c43-ab8f-5ddac012d7ca None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:09.957712 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-b0c6646a-36b1-4c43-ab8f-5ddac012d7ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:04:09.957712 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:04:09.957712 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:04:09.957712 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:04:09.957712 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:04:09.957712 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:04:09.957712 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:04:09.957712 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:04:09.957712 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:04:09.957712 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:04:09.957712 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:04:09.957712 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:04:09.957712 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:04:09.957712 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:04:09.957712 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:04:09.958680 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:04:09.958680 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:04:09.958680 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:04:09.958680 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:04:09.958680 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:04:09.958680 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:04:09.958680 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:04:09.958680 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:04:09.958680 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:04:09.958680 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:04:09.958680 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:04:09.958680 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:04:09.958680 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:04:09.958680 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:04:09.958680 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:04:09.958680 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:04:09.958680 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:04:09.959632 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:04:09.959632 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:04:09.959632 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:04:09.959632 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:04:09.963231 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 515/1041] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:04:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:04:09.964712 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d276b922-f7d8-4af5-813a-b5ab2f831685 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:10.077260 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d276b922-f7d8-4af5-813a-b5ab2f831685 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:10.101213 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:10.112783 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78f066e-e9a3-44bc-9620-3bfbdec81c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:10.113404 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78f066e-e9a3-44bc-9620-3bfbdec81c3e None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:10.113686 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78f066e-e9a3-44bc-9620-3bfbdec81c3e None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:10.113959 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78f066e-e9a3-44bc-9620-3bfbdec81c3e None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:10.114652 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78f066e-e9a3-44bc-9620-3bfbdec81c3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:04:10.114954 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78f066e-e9a3-44bc-9620-3bfbdec81c3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:10.116355 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78f066e-e9a3-44bc-9620-3bfbdec81c3e None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:10.134176 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 527/1042] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:04:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:04:10.135424 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c6646a-36b1-4c43-ab8f-5ddac012d7ca None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:10.166627 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:10.190831 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e428d47-e5e9-4f6a-9b3e-ee095dc33805 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:10.191108 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e428d47-e5e9-4f6a-9b3e-ee095dc33805 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:10.191402 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e428d47-e5e9-4f6a-9b3e-ee095dc33805 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:10.192803 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 528/1043] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:04:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:04:10.197979 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92b1c15-cbe0-448e-8362-528a49bcd291 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:10.198263 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92b1c15-cbe0-448e-8362-528a49bcd291 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:10.198566 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92b1c15-cbe0-448e-8362-528a49bcd291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:10.216739 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-c92b1c15-cbe0-448e-8362-528a49bcd291 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:04:10.218343 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-c92b1c15-cbe0-448e-8362-528a49bcd291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:04:10.246914 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c6646a-36b1-4c43-ab8f-5ddac012d7ca None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:10.282460 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e752e37-51a2-4ff8-bc58-00d0563dc578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:10.283108 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e752e37-51a2-4ff8-bc58-00d0563dc578 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:10.283420 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e752e37-51a2-4ff8-bc58-00d0563dc578 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:10.283696 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e752e37-51a2-4ff8-bc58-00d0563dc578 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:10.290687 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e752e37-51a2-4ff8-bc58-00d0563dc578 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:10.292157 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e752e37-51a2-4ff8-bc58-00d0563dc578 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:10.295365 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 529/1044] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:04:10 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:04:10.304954 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-9e752e37-51a2-4ff8-bc58-00d0563dc578 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:04:10.304954 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:04:10.304954 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:04:10.304954 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:04:10.304954 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:04:10.304954 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:04:10.304954 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:04:10.304954 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:04:10.304954 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:04:10.304954 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:04:10.304954 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:04:10.304954 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:04:10.304954 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 21:04:10.304954 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 21:04:10.304954 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 21:04:10.305837 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 21:04:10.305837 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:04:10.305837 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:04:10.305837 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:04:10.305837 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:04:10.305837 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:04:10.305837 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:04:10.305837 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:04:10.305837 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:04:10.305837 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:04:10.305837 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:04:10.305837 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:04:10.305837 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:04:10.305837 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:04:10.305837 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:04:10.305837 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:04:10.305837 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:04:10.312212 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 21:04:10.312212 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 21:04:10.312212 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 21:04:10.312212 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 21:04:10.312212 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 21:04:10.312212 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 21:04:10.312212 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:04:10.312212 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:04:10.312212 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 516/1045] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Aug 4 21:04:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:04:10.312795 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92b1c15-cbe0-448e-8362-528a49bcd291 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:10.317757 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e752e37-51a2-4ff8-bc58-00d0563dc578 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:10.414195 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:10.417013 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92b1c15-cbe0-448e-8362-528a49bcd291 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:10.424730 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e752e37-51a2-4ff8-bc58-00d0563dc578 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:10.432327 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:10.454035 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08964af-cae7-4932-a08c-29b24466b76f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:10.454651 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08964af-cae7-4932-a08c-29b24466b76f None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:10.454935 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08964af-cae7-4932-a08c-29b24466b76f None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:10.455211 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08964af-cae7-4932-a08c-29b24466b76f None admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:10.455996 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08964af-cae7-4932-a08c-29b24466b76f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:04:10.461608 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08964af-cae7-4932-a08c-29b24466b76f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:10.461608 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08964af-cae7-4932-a08c-29b24466b76f None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:10.461767 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68b0834-afe6-4791-98d6-c151448d14a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:10.462004 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68b0834-afe6-4791-98d6-c151448d14a4 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:10.462289 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68b0834-afe6-4791-98d6-c151448d14a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:10.462560 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68b0834-afe6-4791-98d6-c151448d14a4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:10.474444 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 530/1046] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Aug 4 21:04:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:04:10.475349 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08964af-cae7-4932-a08c-29b24466b76f None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:10.479257 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68b0834-afe6-4791-98d6-c151448d14a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:04:10.484199 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-b68b0834-afe6-4791-98d6-c151448d14a4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:04:10.484199 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:04:10.484199 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:04:10.484199 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:04:10.484199 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:04:10.484199 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:04:10.484199 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:04:10.484199 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:04:10.484199 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:04:10.484199 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:04:10.484199 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:04:10.484199 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:04:10.484199 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:04:10.484199 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:04:10.484199 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:04:10.484199 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:04:10.485119 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:04:10.485119 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:04:10.485119 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:04:10.485119 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:04:10.485119 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:04:10.485119 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:04:10.485119 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:04:10.485119 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:04:10.485119 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:04:10.485119 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:04:10.485119 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:04:10.485119 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:04:10.485119 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:04:10.485119 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:04:10.485119 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:04:10.485119 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:04:10.486522 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:04:10.486522 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:04:10.486522 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:04:10.486522 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:04:10.486522 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:04:10.486522 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:04:10.486522 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:04:10.486522 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:04:10.486522 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:04:10.486522 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:04:10.486522 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:04:10.486522 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:04:10.486522 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 517/1047] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Aug 4 21:04:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:04:10.491873 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68b0834-afe6-4791-98d6-c151448d14a4 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:10.590153 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08964af-cae7-4932-a08c-29b24466b76f None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:10.592762 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68b0834-afe6-4791-98d6-c151448d14a4 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:10.625293 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53712c46-b415-471b-a1c9-7ce2d4088004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:10.625956 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53712c46-b415-471b-a1c9-7ce2d4088004 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:10.626219 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53712c46-b415-471b-a1c9-7ce2d4088004 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:10.626495 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53712c46-b415-471b-a1c9-7ce2d4088004 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:10.627274 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53712c46-b415-471b-a1c9-7ce2d4088004 None admin] RBAC: Adding query filter params (role.id=47f3f6a0aef34d25a88aed7f622b2d46, scope.project.id=17dc3d39e2c048ca9070588f5717a578, user.id=6a2ce1df87fa47e5be301916bf3245fb) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:04:10.627631 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53712c46-b415-471b-a1c9-7ce2d4088004 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:10.628443 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e472954-4a52-4531-8f21-9157ffe66947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:10.629058 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53712c46-b415-471b-a1c9-7ce2d4088004 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:10.629168 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e472954-4a52-4531-8f21-9157ffe66947 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:10.629339 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e472954-4a52-4531-8f21-9157ffe66947 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:10.629611 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e472954-4a52-4531-8f21-9157ffe66947 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:10.630319 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e472954-4a52-4531-8f21-9157ffe66947 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:04:10.630684 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e472954-4a52-4531-8f21-9157ffe66947 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:10.632245 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e472954-4a52-4531-8f21-9157ffe66947 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:10.644225 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 531/1048] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Aug 4 21:04:10 2022] GET /identity/v3/role_assignments?user.id=6a2ce1df87fa47e5be301916bf3245fb&scope.project.id=17dc3d39e2c048ca9070588f5717a578&role.id=47f3f6a0aef34d25a88aed7f622b2d46 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:04:10.647746 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 518/1049] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Aug 4 21:04:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:04:10.653771 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53712c46-b415-471b-a1c9-7ce2d4088004 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:10.699172 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:10.764088 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53712c46-b415-471b-a1c9-7ce2d4088004 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:10.802480 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbe4f01-7fc1-496e-b01a-819e9a01f87e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:10.803137 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbe4f01-7fc1-496e-b01a-819e9a01f87e None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:10.803451 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbe4f01-7fc1-496e-b01a-819e9a01f87e None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:10.803729 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbe4f01-7fc1-496e-b01a-819e9a01f87e None admin] PATH_INFO: `/v3/users/13af542c6bcd45c6b58073c2f623ceba` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:10.817302 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbe4f01-7fc1-496e-b01a-819e9a01f87e None admin] RBAC: Authorizing `identity:get_user(user_id=13af542c6bcd45c6b58073c2f623ceba)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:10.818782 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbe4f01-7fc1-496e-b01a-819e9a01f87e None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:10.820645 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 532/1050] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Aug 4 21:04:10 2022] GET /identity/v3/users/13af542c6bcd45c6b58073c2f623ceba => generated 285 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:04:10.826178 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e472954-4a52-4531-8f21-9157ffe66947 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:10.843630 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:10.932749 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e472954-4a52-4531-8f21-9157ffe66947 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:10.967782 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f76073-ce41-4fae-be85-58f6f8e2ad05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:10.968328 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f76073-ce41-4fae-be85-58f6f8e2ad05 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:10.968609 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f76073-ce41-4fae-be85-58f6f8e2ad05 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:10.968881 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f76073-ce41-4fae-be85-58f6f8e2ad05 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:11.037173 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f76073-ce41-4fae-be85-58f6f8e2ad05 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:11.038636 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f76073-ce41-4fae-be85-58f6f8e2ad05 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:11.053081 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:11.148211 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 519/1051] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Aug 4 21:04:10 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 21:04:11.149712 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbe4f01-7fc1-496e-b01a-819e9a01f87e None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:11.253582 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:11.259804 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbe4f01-7fc1-496e-b01a-819e9a01f87e None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:11.295250 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac2f2e6-18aa-401b-ac02-d07f3522e82e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:11.295912 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac2f2e6-18aa-401b-ac02-d07f3522e82e None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:11.296188 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac2f2e6-18aa-401b-ac02-d07f3522e82e None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:11.296456 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac2f2e6-18aa-401b-ac02-d07f3522e82e None admin] PATH_INFO: `/v3/projects/878f5e86831941d0be014a94c1315f00` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:11.297590 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac2f2e6-18aa-401b-ac02-d07f3522e82e None admin] RBAC: Authorizing `identity:get_project(project_id=878f5e86831941d0be014a94c1315f00)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:11.299038 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac2f2e6-18aa-401b-ac02-d07f3522e82e None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:11.305486 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 533/1052] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Aug 4 21:04:11 2022] GET /identity/v3/projects/878f5e86831941d0be014a94c1315f00 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:04:11.306683 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f76073-ce41-4fae-be85-58f6f8e2ad05 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:11.409103 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f76073-ce41-4fae-be85-58f6f8e2ad05 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:11.423216 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:11.444031 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb95733b-a421-4d85-9ed9-16fbb9a9a4e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:11.444825 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb95733b-a421-4d85-9ed9-16fbb9a9a4e0 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:11.445110 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb95733b-a421-4d85-9ed9-16fbb9a9a4e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:11.445436 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb95733b-a421-4d85-9ed9-16fbb9a9a4e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:11.446449 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb95733b-a421-4d85-9ed9-16fbb9a9a4e0 None admin] RBAC: Adding query filter params (role.id=a432dd4735944ecfab2fa5ce1705d356, scope.project.id=878f5e86831941d0be014a94c1315f00, user.id=13af542c6bcd45c6b58073c2f623ceba) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:04:11.446781 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb95733b-a421-4d85-9ed9-16fbb9a9a4e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:11.448253 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb95733b-a421-4d85-9ed9-16fbb9a9a4e0 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:11.459350 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 520/1053] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Aug 4 21:04:11 2022] GET /identity/v3/role_assignments?user.id=13af542c6bcd45c6b58073c2f623ceba&scope.project.id=878f5e86831941d0be014a94c1315f00&role.id=a432dd4735944ecfab2fa5ce1705d356 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:04:11.840915 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e83cf1-8c24-4978-bb7a-9cd1f9d7d5dc None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:11.841465 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e83cf1-8c24-4978-bb7a-9cd1f9d7d5dc None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:11.841465 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e83cf1-8c24-4978-bb7a-9cd1f9d7d5dc None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:11.842933 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 534/1054] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:04:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:04:11.847948 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa181621-29a9-461c-9251-789d0d019f4d None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:11.848326 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa181621-29a9-461c-9251-789d0d019f4d None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:11.848501 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa181621-29a9-461c-9251-789d0d019f4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:11.866788 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-fa181621-29a9-461c-9251-789d0d019f4d None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:04:11.868001 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-fa181621-29a9-461c-9251-789d0d019f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:04:11.935141 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 521/1055] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:04:11 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:04:11.957803 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e83cf1-8c24-4978-bb7a-9cd1f9d7d5dc None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:12.061169 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e83cf1-8c24-4978-bb7a-9cd1f9d7d5dc None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:12.095230 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd652f95-1b0d-4059-a166-94284b55ffb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:12.095824 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd652f95-1b0d-4059-a166-94284b55ffb7 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:12.096109 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd652f95-1b0d-4059-a166-94284b55ffb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:12.096368 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd652f95-1b0d-4059-a166-94284b55ffb7 None admin] PATH_INFO: `/v3/services/image` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:12.096992 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd652f95-1b0d-4059-a166-94284b55ffb7 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:12.098350 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd652f95-1b0d-4059-a166-94284b55ffb7 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:12.107199 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-bd652f95-1b0d-4059-a166-94284b55ffb7 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 21:04:12.107199 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:04:12.107199 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 21:04:12.107199 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 21:04:12.107199 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 21:04:12.107199 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 21:04:12.107199 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 21:04:12.107199 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:04:12.107199 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 21:04:12.107199 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:04:12.107199 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 21:04:12.107199 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:04:12.107199 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:04:12.107199 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:04:12.107199 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:04:12.107199 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:04:12.107199 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:04:12.108301 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:04:12.108301 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:04:12.108301 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:04:12.108301 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:04:12.108301 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:04:12.108301 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:04:12.108301 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 21:04:12.108301 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 21:04:12.108301 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 21:04:12.108301 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 21:04:12.108301 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:04:12.108301 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:04:12.108301 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:04:12.108301 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:04:12.108301 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:04:12.108301 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:04:12.109162 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:04:12.109162 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:04:12.109162 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:04:12.109162 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:04:12.109162 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:04:12.109162 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:04:12.109162 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:04:12.109162 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:04:12.109162 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:04:12.109162 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:04:12.109162 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 21:04:12.109162 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:04:12.109162 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 21:04:12.109162 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:04:12.111636 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 535/1056] 10.222.0.25 () {56 vars in 1105 bytes} [Thu Aug 4 21:04:11 2022] GET /identity/v3/services/image => generated 86 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:04:12.114858 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa181621-29a9-461c-9251-789d0d019f4d None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:12.213764 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa181621-29a9-461c-9251-789d0d019f4d None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:12.229303 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:12.254362 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7b2c5e-a8af-4d2b-b16f-e71111d2d09e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:12.254991 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7b2c5e-a8af-4d2b-b16f-e71111d2d09e None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:12.255275 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7b2c5e-a8af-4d2b-b16f-e71111d2d09e None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:12.255583 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7b2c5e-a8af-4d2b-b16f-e71111d2d09e None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:12.256268 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7b2c5e-a8af-4d2b-b16f-e71111d2d09e None admin] RBAC: Adding query filter params (name=image) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:04:12.256574 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7b2c5e-a8af-4d2b-b16f-e71111d2d09e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:12.257943 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7b2c5e-a8af-4d2b-b16f-e71111d2d09e None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:12.263516 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 522/1057] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:04:12 2022] GET /identity/v3/services?name=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:04:12.269000 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd652f95-1b0d-4059-a166-94284b55ffb7 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:12.298924 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:12.370251 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd652f95-1b0d-4059-a166-94284b55ffb7 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:12.403889 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0b3a14-4049-417e-a673-735f93e2692a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:12.404515 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0b3a14-4049-417e-a673-735f93e2692a None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:12.404800 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0b3a14-4049-417e-a673-735f93e2692a None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:12.405077 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0b3a14-4049-417e-a673-735f93e2692a None admin] PATH_INFO: `/v3/services` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:12.405759 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0b3a14-4049-417e-a673-735f93e2692a None admin] RBAC: Adding query filter params (type=image) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:04:12.406062 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0b3a14-4049-417e-a673-735f93e2692a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:12.408081 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0b3a14-4049-417e-a673-735f93e2692a None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:12.419292 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 536/1058] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:04:12 2022] GET /identity/v3/services?type=image => generated 350 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:04:12.420551 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7b2c5e-a8af-4d2b-b16f-e71111d2d09e None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:12.521326 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7b2c5e-a8af-4d2b-b16f-e71111d2d09e None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:12.557587 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471dd376-9075-4052-950d-4558ef42da8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:12.558215 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471dd376-9075-4052-950d-4558ef42da8d None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:12.558498 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471dd376-9075-4052-950d-4558ef42da8d None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:12.558774 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471dd376-9075-4052-950d-4558ef42da8d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:12.559529 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471dd376-9075-4052-950d-4558ef42da8d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:12.560902 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471dd376-9075-4052-950d-4558ef42da8d None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:12.634987 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7fe07e-878c-4f7f-bfc7-38c2393bd5db None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:12.635196 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7fe07e-878c-4f7f-bfc7-38c2393bd5db None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:12.635510 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7fe07e-878c-4f7f-bfc7-38c2393bd5db None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:12.637111 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 537/1059] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:04:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 21:04:12.642625 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4542b9-fe23-42fd-b2ab-1c3d885918f9 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:12.643070 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4542b9-fe23-42fd-b2ab-1c3d885918f9 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:12.643429 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4542b9-fe23-42fd-b2ab-1c3d885918f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:12.645772 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 523/1060] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:04:12 2022] POST /identity/v3/endpoints => generated 327 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:04:12.668930 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-7e4542b9-fe23-42fd-b2ab-1c3d885918f9 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:04:12.670153 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-7e4542b9-fe23-42fd-b2ab-1c3d885918f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:04:12.749802 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 538/1061] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:04:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 21:04:12.773101 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471dd376-9075-4052-950d-4558ef42da8d None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:12.804376 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:12.880834 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471dd376-9075-4052-950d-4558ef42da8d None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:12.916093 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a09f1b9-b8e9-4778-81d8-7a524e5c52ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:12.916749 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a09f1b9-b8e9-4778-81d8-7a524e5c52ff None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:12.917033 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a09f1b9-b8e9-4778-81d8-7a524e5c52ff None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:12.917308 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a09f1b9-b8e9-4778-81d8-7a524e5c52ff None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:12.927409 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a09f1b9-b8e9-4778-81d8-7a524e5c52ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:04:12.932297 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-4a09f1b9-b8e9-4778-81d8-7a524e5c52ff None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 21:04:12.932297 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:04:12.932297 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:04:12.932297 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:04:12.932297 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:04:12.932297 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:04:12.932297 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:04:12.932297 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:04:12.932297 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:04:12.932297 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:04:12.932297 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:04:12.932297 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:04:12.932297 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:04:12.932297 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:04:12.932297 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:04:12.932297 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:04:12.933244 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:04:12.933244 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:04:12.933244 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:04:12.933244 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:04:12.933244 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:04:12.933244 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:04:12.933244 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:04:12.933244 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:04:12.933244 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:04:12.933244 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:04:12.933244 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:04:12.933244 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:04:12.933244 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:04:12.933244 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:04:12.933244 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:04:12.933244 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:04:12.934117 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:04:12.934117 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:04:12.934117 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:04:12.934117 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:04:12.934117 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:04:12.934117 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:04:12.934117 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:04:12.934117 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:04:12.934117 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:04:12.934117 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:04:12.934117 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 21:04:12.934117 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:04:12.934117 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 524/1062] 10.222.0.25 () {56 vars in 1120 bytes} [Thu Aug 4 21:04:12 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:04:12.939903 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4542b9-fe23-42fd-b2ab-1c3d885918f9 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:12.959613 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:13.043618 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4542b9-fe23-42fd-b2ab-1c3d885918f9 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:13.079948 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57c4b94-c0c4-4c57-acf2-0bdfdf585955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:13.080583 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57c4b94-c0c4-4c57-acf2-0bdfdf585955 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:13.080875 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57c4b94-c0c4-4c57-acf2-0bdfdf585955 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:13.081142 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57c4b94-c0c4-4c57-acf2-0bdfdf585955 None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:13.081853 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57c4b94-c0c4-4c57-acf2-0bdfdf585955 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:04:13.082156 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57c4b94-c0c4-4c57-acf2-0bdfdf585955 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:13.083549 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57c4b94-c0c4-4c57-acf2-0bdfdf585955 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:13.091886 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 539/1063] 10.222.0.25 () {56 vars in 1115 bytes} [Thu Aug 4 21:04:12 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:04:13.097598 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a09f1b9-b8e9-4778-81d8-7a524e5c52ff None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:13.187289 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:13.200088 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a09f1b9-b8e9-4778-81d8-7a524e5c52ff None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:13.240522 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0329f4b0-60ca-47e0-b6dd-b0d850ea07b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:13.241172 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0329f4b0-60ca-47e0-b6dd-b0d850ea07b3 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:13.241454 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0329f4b0-60ca-47e0-b6dd-b0d850ea07b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:13.241727 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0329f4b0-60ca-47e0-b6dd-b0d850ea07b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:13.252412 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0329f4b0-60ca-47e0-b6dd-b0d850ea07b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:13.253874 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0329f4b0-60ca-47e0-b6dd-b0d850ea07b3 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:13.268301 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-0329f4b0-60ca-47e0-b6dd-b0d850ea07b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:04:13.268301 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:04:13.268301 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:04:13.268301 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:04:13.268301 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:04:13.268301 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:04:13.268301 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:04:13.268301 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:04:13.268301 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:04:13.268301 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:04:13.268301 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:04:13.268301 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:04:13.268301 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:04:13.268301 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:04:13.268301 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:04:13.269191 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:04:13.269191 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:04:13.269191 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:04:13.269191 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:04:13.269191 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:04:13.269191 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:04:13.269191 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:04:13.269191 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:04:13.269191 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:04:13.269191 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:04:13.269191 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:04:13.269191 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:04:13.269191 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:04:13.269191 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:04:13.269191 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:04:13.269191 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:04:13.269191 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:04:13.270078 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:04:13.270078 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:04:13.270078 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:04:13.270078 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:04:13.270851 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 525/1064] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:04:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:04:13.275096 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57c4b94-c0c4-4c57-acf2-0bdfdf585955 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:13.356145 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:13.376878 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57c4b94-c0c4-4c57-acf2-0bdfdf585955 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:13.411929 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47568d44-4497-4ae5-991d-92aed9a9ba6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:13.412553 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47568d44-4497-4ae5-991d-92aed9a9ba6c None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:13.412834 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47568d44-4497-4ae5-991d-92aed9a9ba6c None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:13.413105 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47568d44-4497-4ae5-991d-92aed9a9ba6c None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:13.413791 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47568d44-4497-4ae5-991d-92aed9a9ba6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:04:13.414094 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47568d44-4497-4ae5-991d-92aed9a9ba6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:13.415467 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47568d44-4497-4ae5-991d-92aed9a9ba6c None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:13.427593 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 540/1065] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:04:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:04:13.432983 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0329f4b0-60ca-47e0-b6dd-b0d850ea07b3 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:13.435874 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00797c67-301a-44cf-a7bb-0a2416b63a0a None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:13.436194 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00797c67-301a-44cf-a7bb-0a2416b63a0a None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:13.436502 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00797c67-301a-44cf-a7bb-0a2416b63a0a None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:13.437923 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 541/1066] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:04:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 21:04:13.443911 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43101095-a26f-412c-b2e8-e03895484612 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:13.444200 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43101095-a26f-412c-b2e8-e03895484612 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:13.444476 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43101095-a26f-412c-b2e8-e03895484612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:13.463230 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-43101095-a26f-412c-b2e8-e03895484612 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:04:13.464676 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-43101095-a26f-412c-b2e8-e03895484612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:04:13.534032 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 542/1067] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:04:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 21:04:13.543193 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0329f4b0-60ca-47e0-b6dd-b0d850ea07b3 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:13.556636 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43101095-a26f-412c-b2e8-e03895484612 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:13.581223 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37993cd3-196e-45b3-93ff-f4596aa8c19d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:13.581947 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37993cd3-196e-45b3-93ff-f4596aa8c19d None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:13.582302 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37993cd3-196e-45b3-93ff-f4596aa8c19d None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:13.582651 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37993cd3-196e-45b3-93ff-f4596aa8c19d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:13.590171 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37993cd3-196e-45b3-93ff-f4596aa8c19d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:13.591720 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37993cd3-196e-45b3-93ff-f4596aa8c19d None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:13.595265 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:13.604994 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-37993cd3-196e-45b3-93ff-f4596aa8c19d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 21:04:13.604994 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:04:13.604994 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:04:13.604994 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:04:13.604994 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:04:13.604994 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:04:13.604994 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:04:13.604994 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:04:13.604994 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:04:13.604994 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:04:13.604994 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:04:13.604994 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:04:13.604994 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 21:04:13.604994 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 21:04:13.604994 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 21:04:13.604994 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 21:04:13.606124 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:04:13.606124 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:04:13.606124 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 21:04:13.606124 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:04:13.606124 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 21:04:13.606124 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:04:13.606124 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:04:13.606124 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:04:13.606124 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:04:13.606124 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:04:13.606124 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:04:13.606124 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:04:13.606124 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:04:13.606124 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:04:13.606124 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:04:13.606124 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:04:13.606124 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:04:13.606124 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:04:13.607258 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:04:13.607258 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:04:13.607258 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 21:04:13.607258 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 21:04:13.607258 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 21:04:13.607258 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 21:04:13.607258 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 21:04:13.607258 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 21:04:13.607258 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 21:04:13.607258 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:04:13.608240 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 526/1068] 10.222.0.25 () {56 vars in 1093 bytes} [Thu Aug 4 21:04:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:04:13.614060 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37993cd3-196e-45b3-93ff-f4596aa8c19d None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:13.634696 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:13.668028 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43101095-a26f-412c-b2e8-e03895484612 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:13.703334 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca89cae-d20d-43f8-b6a4-4d5ad2eb8ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:13.703935 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca89cae-d20d-43f8-b6a4-4d5ad2eb8ab6 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:13.704217 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca89cae-d20d-43f8-b6a4-4d5ad2eb8ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:13.704491 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca89cae-d20d-43f8-b6a4-4d5ad2eb8ab6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:13.716121 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca89cae-d20d-43f8-b6a4-4d5ad2eb8ab6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:04:13.719140 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37993cd3-196e-45b3-93ff-f4596aa8c19d None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:13.721819 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-dca89cae-d20d-43f8-b6a4-4d5ad2eb8ab6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:04:13.721819 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:04:13.721819 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:04:13.721819 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:04:13.721819 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:04:13.721819 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:04:13.721819 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:04:13.721819 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:04:13.721819 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:04:13.721819 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:04:13.721819 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:04:13.721819 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:04:13.721819 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:04:13.721819 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:04:13.721819 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:04:13.721819 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:04:13.722838 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:04:13.722838 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:04:13.722838 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:04:13.722838 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:04:13.722838 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:04:13.722838 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:04:13.722838 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:04:13.722838 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:04:13.722838 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:04:13.722838 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:04:13.722838 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:04:13.722838 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:04:13.722838 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:04:13.722838 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:04:13.722838 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:04:13.722838 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:04:13.723698 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:04:13.723698 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:04:13.723698 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:04:13.723698 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:04:13.723698 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:04:13.723698 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:04:13.723698 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:04:13.723698 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:04:13.723698 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:04:13.723698 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:04:13.723698 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:04:13.723698 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:04:13.724343 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 543/1069] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Aug 4 21:04:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:04:13.731647 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca89cae-d20d-43f8-b6a4-4d5ad2eb8ab6 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:13.766074 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173e730b-83be-44a1-b455-b1deb3a2c92a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:13.766733 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173e730b-83be-44a1-b455-b1deb3a2c92a None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:13.767181 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173e730b-83be-44a1-b455-b1deb3a2c92a None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:13.767494 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173e730b-83be-44a1-b455-b1deb3a2c92a None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:13.768874 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173e730b-83be-44a1-b455-b1deb3a2c92a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:04:13.769189 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173e730b-83be-44a1-b455-b1deb3a2c92a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:13.770794 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173e730b-83be-44a1-b455-b1deb3a2c92a None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:13.794929 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 527/1070] 10.222.0.25 () {56 vars in 1133 bytes} [Thu Aug 4 21:04:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:04:13.800351 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173e730b-83be-44a1-b455-b1deb3a2c92a None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:13.847352 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca89cae-d20d-43f8-b6a4-4d5ad2eb8ab6 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:13.862602 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:13.878858 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:13.885082 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e26cbc-066b-4a97-bdad-ca6ff348039f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:13.885698 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e26cbc-066b-4a97-bdad-ca6ff348039f None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:13.886020 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e26cbc-066b-4a97-bdad-ca6ff348039f None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:13.886331 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e26cbc-066b-4a97-bdad-ca6ff348039f None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:13.887042 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e26cbc-066b-4a97-bdad-ca6ff348039f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:04:13.887362 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e26cbc-066b-4a97-bdad-ca6ff348039f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:13.888898 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e26cbc-066b-4a97-bdad-ca6ff348039f None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:13.903068 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 544/1071] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Aug 4 21:04:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:04:13.903897 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e26cbc-066b-4a97-bdad-ca6ff348039f None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:13.913045 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173e730b-83be-44a1-b455-b1deb3a2c92a None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:13.949489 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f891687-037f-411c-887c-3ed8497a9823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:13.950580 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f891687-037f-411c-887c-3ed8497a9823 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:13.950935 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f891687-037f-411c-887c-3ed8497a9823 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:13.951565 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f891687-037f-411c-887c-3ed8497a9823 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:13.965682 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f891687-037f-411c-887c-3ed8497a9823 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:13.967481 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f891687-037f-411c-887c-3ed8497a9823 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:13.986386 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-8f891687-037f-411c-887c-3ed8497a9823 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:04:13.986386 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:04:13.986386 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:04:13.986386 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:04:13.986386 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:04:13.986386 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:04:13.986386 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:04:13.986386 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:04:13.986386 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:04:13.986386 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:04:13.986386 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:04:13.986386 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:04:13.986386 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:04:13.986386 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:04:13.986386 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:04:13.987499 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:04:13.987499 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:04:13.987499 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:04:13.987499 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:04:13.987499 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:04:13.987499 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:04:13.987499 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:04:13.987499 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:04:13.987499 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:04:13.987499 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:04:13.987499 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:04:13.987499 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:04:13.987499 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:04:13.987499 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:04:13.987499 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:04:13.987499 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:04:13.987499 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:04:13.988598 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:04:13.988598 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:04:13.988598 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:04:13.988598 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:04:13.991460 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 528/1072] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:04:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:04:13.993413 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f891687-037f-411c-887c-3ed8497a9823 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:14.010271 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e26cbc-066b-4a97-bdad-ca6ff348039f None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:14.047424 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6bfaec-4c7e-422d-86ba-ec37ae7d5c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:14.048589 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6bfaec-4c7e-422d-86ba-ec37ae7d5c49 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:14.048873 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6bfaec-4c7e-422d-86ba-ec37ae7d5c49 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:14.049142 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6bfaec-4c7e-422d-86ba-ec37ae7d5c49 None admin] PATH_INFO: `/v3/users/13af542c6bcd45c6b58073c2f623ceba` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:14.050406 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6bfaec-4c7e-422d-86ba-ec37ae7d5c49 None admin] RBAC: Authorizing `identity:get_user(user_id=13af542c6bcd45c6b58073c2f623ceba)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:14.051992 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6bfaec-4c7e-422d-86ba-ec37ae7d5c49 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:14.054007 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 545/1073] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Aug 4 21:04:13 2022] GET /identity/v3/users/13af542c6bcd45c6b58073c2f623ceba => generated 285 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:04:14.059346 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6bfaec-4c7e-422d-86ba-ec37ae7d5c49 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:14.096240 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f891687-037f-411c-887c-3ed8497a9823 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:14.106749 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:14.119467 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:14.132928 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82bdc80-ec82-41ca-83f4-3e8edbbef95b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:14.133555 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82bdc80-ec82-41ca-83f4-3e8edbbef95b None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:14.133835 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82bdc80-ec82-41ca-83f4-3e8edbbef95b None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:14.134106 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82bdc80-ec82-41ca-83f4-3e8edbbef95b None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:14.134852 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82bdc80-ec82-41ca-83f4-3e8edbbef95b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:04:14.135192 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82bdc80-ec82-41ca-83f4-3e8edbbef95b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:14.136714 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82bdc80-ec82-41ca-83f4-3e8edbbef95b None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:14.149543 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 529/1074] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:04:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:04:14.154979 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82bdc80-ec82-41ca-83f4-3e8edbbef95b None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:14.163133 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6bfaec-4c7e-422d-86ba-ec37ae7d5c49 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:14.199645 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac8cea2-ad2c-4dde-8ef0-975cad9be83b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:14.200485 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac8cea2-ad2c-4dde-8ef0-975cad9be83b None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:14.200840 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac8cea2-ad2c-4dde-8ef0-975cad9be83b None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:14.201184 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac8cea2-ad2c-4dde-8ef0-975cad9be83b None admin] PATH_INFO: `/v3/projects/878f5e86831941d0be014a94c1315f00` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:14.202415 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac8cea2-ad2c-4dde-8ef0-975cad9be83b None admin] RBAC: Authorizing `identity:get_project(project_id=878f5e86831941d0be014a94c1315f00)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:14.203981 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac8cea2-ad2c-4dde-8ef0-975cad9be83b None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:14.210509 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 546/1075] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Aug 4 21:04:14 2022] GET /identity/v3/projects/878f5e86831941d0be014a94c1315f00 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:04:14.211618 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac8cea2-ad2c-4dde-8ef0-975cad9be83b None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:14.269964 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82bdc80-ec82-41ca-83f4-3e8edbbef95b None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:14.307900 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691b130b-542e-4eeb-9b23-08ad8784e36a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:14.308463 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b130b-542e-4eeb-9b23-08ad8784e36a None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:14.308743 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b130b-542e-4eeb-9b23-08ad8784e36a None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:14.309013 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b130b-542e-4eeb-9b23-08ad8784e36a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:14.315881 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691b130b-542e-4eeb-9b23-08ad8784e36a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:14.317333 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691b130b-542e-4eeb-9b23-08ad8784e36a None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:14.325628 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac8cea2-ad2c-4dde-8ef0-975cad9be83b None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:14.328409 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-691b130b-542e-4eeb-9b23-08ad8784e36a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:04:14.328409 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:04:14.328409 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:04:14.328409 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:04:14.328409 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:04:14.328409 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:04:14.328409 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:04:14.328409 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:04:14.328409 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:04:14.328409 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:04:14.328409 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:04:14.328409 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:04:14.328409 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 21:04:14.328409 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 21:04:14.328409 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 21:04:14.329374 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 21:04:14.329374 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:04:14.329374 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:04:14.329374 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:04:14.329374 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:04:14.329374 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:04:14.329374 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:04:14.329374 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:04:14.329374 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:04:14.329374 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:04:14.329374 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:04:14.329374 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:04:14.329374 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:04:14.329374 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:04:14.329374 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:04:14.329374 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:04:14.329374 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:04:14.330251 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 21:04:14.330251 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 21:04:14.330251 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 21:04:14.330251 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 21:04:14.330251 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 21:04:14.330251 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 21:04:14.330251 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:04:14.330251 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:04:14.330251 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 530/1076] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Aug 4 21:04:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:04:14.335264 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691b130b-542e-4eeb-9b23-08ad8784e36a None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:14.356860 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:14.365044 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1a5cee-88d6-48b2-94ca-1cfa37c37755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:14.365723 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1a5cee-88d6-48b2-94ca-1cfa37c37755 None admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:14.369017 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1a5cee-88d6-48b2-94ca-1cfa37c37755 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:14.369017 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1a5cee-88d6-48b2-94ca-1cfa37c37755 None admin] PATH_INFO: `/v3/projects/878f5e86831941d0be014a94c1315f00/users/13af542c6bcd45c6b58073c2f623ceba/roles/a432dd4735944ecfab2fa5ce1705d356` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:14.369017 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1a5cee-88d6-48b2-94ca-1cfa37c37755 None admin] RBAC: Authorizing `identity:create_grant(project_id=878f5e86831941d0be014a94c1315f00, user_id=13af542c6bcd45c6b58073c2f623ceba, role_id=a432dd4735944ecfab2fa5ce1705d356)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:14.369797 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1a5cee-88d6-48b2-94ca-1cfa37c37755 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:14.370845 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:14.445705 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691b130b-542e-4eeb-9b23-08ad8784e36a None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:14.475596 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 547/1077] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Aug 4 21:04:14 2022] PUT /identity/v3/projects/878f5e86831941d0be014a94c1315f00/users/13af542c6bcd45c6b58073c2f623ceba/roles/a432dd4735944ecfab2fa5ce1705d356 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:04:14.488373 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85184945-2d78-4d3b-869f-0523bdcb7d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:14.489049 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85184945-2d78-4d3b-869f-0523bdcb7d7b None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:14.489342 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85184945-2d78-4d3b-869f-0523bdcb7d7b None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:14.489708 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85184945-2d78-4d3b-869f-0523bdcb7d7b None admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:14.490487 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85184945-2d78-4d3b-869f-0523bdcb7d7b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:04:14.490795 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85184945-2d78-4d3b-869f-0523bdcb7d7b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:14.492969 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85184945-2d78-4d3b-869f-0523bdcb7d7b None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:14.514697 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 531/1078] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Aug 4 21:04:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:04:14.516521 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1a5cee-88d6-48b2-94ca-1cfa37c37755 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:14.638674 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1a5cee-88d6-48b2-94ca-1cfa37c37755 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:14.677622 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f59cdb5-0414-4cd8-bb00-3a1b85e358b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:14.678300 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f59cdb5-0414-4cd8-bb00-3a1b85e358b7 None admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:14.678582 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f59cdb5-0414-4cd8-bb00-3a1b85e358b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:14.678855 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f59cdb5-0414-4cd8-bb00-3a1b85e358b7 None admin] PATH_INFO: `/v3/projects/17dc3d39e2c048ca9070588f5717a578/users/6a2ce1df87fa47e5be301916bf3245fb/roles/47f3f6a0aef34d25a88aed7f622b2d46` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:14.681014 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f59cdb5-0414-4cd8-bb00-3a1b85e358b7 None admin] RBAC: Authorizing `identity:create_grant(project_id=17dc3d39e2c048ca9070588f5717a578, user_id=6a2ce1df87fa47e5be301916bf3245fb, role_id=47f3f6a0aef34d25a88aed7f622b2d46)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:14.683765 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f59cdb5-0414-4cd8-bb00-3a1b85e358b7 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:14.753812 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766a800c-e3ee-4353-99a1-01b097defd82 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:14.754000 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766a800c-e3ee-4353-99a1-01b097defd82 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:14.754314 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766a800c-e3ee-4353-99a1-01b097defd82 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:14.755728 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 532/1079] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:04:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:04:14.759882 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 548/1080] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Aug 4 21:04:14 2022] PUT /identity/v3/projects/17dc3d39e2c048ca9070588f5717a578/users/6a2ce1df87fa47e5be301916bf3245fb/roles/47f3f6a0aef34d25a88aed7f622b2d46 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:04:14.760737 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feffa699-9d34-44ad-a34d-1b640f78eeab None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:14.761028 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feffa699-9d34-44ad-a34d-1b640f78eeab None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:14.761307 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feffa699-9d34-44ad-a34d-1b640f78eeab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:14.782187 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-feffa699-9d34-44ad-a34d-1b640f78eeab None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:04:14.783448 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-feffa699-9d34-44ad-a34d-1b640f78eeab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:04:14.869965 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 533/1081] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:04:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 21:04:14.893803 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f59cdb5-0414-4cd8-bb00-3a1b85e358b7 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:14.953560 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:15.013298 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f59cdb5-0414-4cd8-bb00-3a1b85e358b7 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:15.058532 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fd8b3d-5f0c-4e1d-bfac-a191d4fe8368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:15.059165 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd8b3d-5f0c-4e1d-bfac-a191d4fe8368 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:15.059476 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd8b3d-5f0c-4e1d-bfac-a191d4fe8368 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:15.059751 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd8b3d-5f0c-4e1d-bfac-a191d4fe8368 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:15.139686 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fd8b3d-5f0c-4e1d-bfac-a191d4fe8368 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:15.141155 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fd8b3d-5f0c-4e1d-bfac-a191d4fe8368 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:15.196148 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:15.259605 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 549/1082] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Aug 4 21:04:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 21:04:15.265423 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feffa699-9d34-44ad-a34d-1b640f78eeab None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:15.287999 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:15.364976 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feffa699-9d34-44ad-a34d-1b640f78eeab None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:15.398134 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369d52a0-67fe-4e21-9367-c2d54585eb8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:15.398778 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369d52a0-67fe-4e21-9367-c2d54585eb8d None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:15.399062 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369d52a0-67fe-4e21-9367-c2d54585eb8d None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:15.399359 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369d52a0-67fe-4e21-9367-c2d54585eb8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:15.409301 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369d52a0-67fe-4e21-9367-c2d54585eb8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:15.411002 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369d52a0-67fe-4e21-9367-c2d54585eb8d None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:15.425988 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-369d52a0-67fe-4e21-9367-c2d54585eb8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:04:15.425988 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:04:15.425988 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:04:15.425988 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:04:15.425988 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:04:15.425988 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:04:15.425988 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:04:15.425988 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:04:15.425988 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:04:15.425988 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:04:15.425988 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:04:15.425988 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:04:15.425988 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:04:15.425988 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:04:15.425988 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:04:15.426880 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:04:15.426880 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:04:15.426880 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:04:15.426880 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:04:15.426880 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:04:15.426880 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:04:15.426880 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:04:15.426880 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:04:15.426880 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:04:15.426880 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:04:15.426880 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:04:15.426880 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:04:15.426880 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:04:15.426880 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:04:15.426880 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:04:15.426880 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:04:15.426880 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:04:15.427827 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:04:15.427827 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:04:15.427827 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:04:15.427827 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:04:15.431375 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 534/1083] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:04:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:04:15.433092 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fd8b3d-5f0c-4e1d-bfac-a191d4fe8368 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:15.532365 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fd8b3d-5f0c-4e1d-bfac-a191d4fe8368 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:15.568295 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a05322-3e6b-4690-be74-db683079680a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:15.568922 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a05322-3e6b-4690-be74-db683079680a None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:15.569207 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a05322-3e6b-4690-be74-db683079680a None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:15.569484 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a05322-3e6b-4690-be74-db683079680a None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:15.570176 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a05322-3e6b-4690-be74-db683079680a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:04:15.570486 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a05322-3e6b-4690-be74-db683079680a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:15.571922 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a05322-3e6b-4690-be74-db683079680a None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:15.591353 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 550/1084] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:04:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:04:16.581094 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729a519c-6981-4ee8-bc6c-8e7d25e0c08b None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:16.581592 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729a519c-6981-4ee8-bc6c-8e7d25e0c08b None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:16.581592 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729a519c-6981-4ee8-bc6c-8e7d25e0c08b None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:16.582927 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 535/1085] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:04:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:04:16.588203 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194c7bf5-8c62-468f-847a-418962a04f76 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:16.588613 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194c7bf5-8c62-468f-847a-418962a04f76 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:16.588764 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194c7bf5-8c62-468f-847a-418962a04f76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:16.607088 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-194c7bf5-8c62-468f-847a-418962a04f76 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:04:16.608290 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-194c7bf5-8c62-468f-847a-418962a04f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:04:16.610217 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:16.675406 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 551/1086] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:04:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:04:16.702125 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729a519c-6981-4ee8-bc6c-8e7d25e0c08b None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:16.778685 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:16.804620 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e3900f-1a5f-43f8-9766-cacb8da955b6 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:16.804926 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e3900f-1a5f-43f8-9766-cacb8da955b6 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:16.805247 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e3900f-1a5f-43f8-9766-cacb8da955b6 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:16.806589 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 552/1087] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:04:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 21:04:16.808087 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729a519c-6981-4ee8-bc6c-8e7d25e0c08b None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:16.811903 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe89c109-bcdb-41f9-bd48-5bdc313af90a None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:16.812184 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe89c109-bcdb-41f9-bd48-5bdc313af90a None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:16.812455 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe89c109-bcdb-41f9-bd48-5bdc313af90a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:16.830816 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-fe89c109-bcdb-41f9-bd48-5bdc313af90a None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:04:16.832066 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-fe89c109-bcdb-41f9-bd48-5bdc313af90a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:04:16.843539 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b733f8-4862-4426-844a-861c82baf6a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:16.844178 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b733f8-4862-4426-844a-861c82baf6a4 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:16.844481 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b733f8-4862-4426-844a-861c82baf6a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:16.844756 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b733f8-4862-4426-844a-861c82baf6a4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:16.855393 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b733f8-4862-4426-844a-861c82baf6a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:04:16.860744 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-f0b733f8-4862-4426-844a-861c82baf6a4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:04:16.860744 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:04:16.860744 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:04:16.860744 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:04:16.860744 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:04:16.860744 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:04:16.860744 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:04:16.860744 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:04:16.860744 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:04:16.860744 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:04:16.860744 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:04:16.860744 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:04:16.860744 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:04:16.860744 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:04:16.860744 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:04:16.860744 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:04:16.861854 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:04:16.861854 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:04:16.861854 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:04:16.861854 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:04:16.861854 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:04:16.861854 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:04:16.861854 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:04:16.861854 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:04:16.861854 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:04:16.861854 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:04:16.861854 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:04:16.861854 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:04:16.861854 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:04:16.861854 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:04:16.861854 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:04:16.861854 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:04:16.862722 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:04:16.862722 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:04:16.862722 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:04:16.862722 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:04:16.862722 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:04:16.862722 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:04:16.862722 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:04:16.862722 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:04:16.862722 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:04:16.862722 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:04:16.862722 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:04:16.862722 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:04:16.862722 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 536/1088] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Aug 4 21:04:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:04:16.868110 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b733f8-4862-4426-844a-861c82baf6a4 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:16.900592 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 553/1089] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:04:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:04:16.923994 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe89c109-bcdb-41f9-bd48-5bdc313af90a None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:16.967406 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b733f8-4862-4426-844a-861c82baf6a4 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:16.988818 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:16.995977 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:17.000742 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107dd557-d6d6-4b51-a98c-fc9bcfaa13f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:17.001930 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107dd557-d6d6-4b51-a98c-fc9bcfaa13f6 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:17.002211 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107dd557-d6d6-4b51-a98c-fc9bcfaa13f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:17.002483 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107dd557-d6d6-4b51-a98c-fc9bcfaa13f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:17.003183 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107dd557-d6d6-4b51-a98c-fc9bcfaa13f6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:04:17.003573 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107dd557-d6d6-4b51-a98c-fc9bcfaa13f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:17.004942 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107dd557-d6d6-4b51-a98c-fc9bcfaa13f6 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:17.013243 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 537/1090] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Aug 4 21:04:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:04:17.019107 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107dd557-d6d6-4b51-a98c-fc9bcfaa13f6 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:17.022655 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe89c109-bcdb-41f9-bd48-5bdc313af90a None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:17.058810 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1030fc93-325f-490e-b050-bf12eee2dc30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:17.059489 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1030fc93-325f-490e-b050-bf12eee2dc30 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:17.059766 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1030fc93-325f-490e-b050-bf12eee2dc30 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:17.060042 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1030fc93-325f-490e-b050-bf12eee2dc30 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:17.070992 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1030fc93-325f-490e-b050-bf12eee2dc30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:04:17.075907 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-1030fc93-325f-490e-b050-bf12eee2dc30 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 21:04:17.075907 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:04:17.075907 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:04:17.075907 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:04:17.075907 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:04:17.075907 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:04:17.075907 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:04:17.075907 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:04:17.075907 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:04:17.075907 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:04:17.075907 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:04:17.075907 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:04:17.075907 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:04:17.075907 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:04:17.075907 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:04:17.075907 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:04:17.076873 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:04:17.076873 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:04:17.076873 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:04:17.076873 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:04:17.076873 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:04:17.076873 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:04:17.076873 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:04:17.076873 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:04:17.076873 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:04:17.076873 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:04:17.076873 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:04:17.076873 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:04:17.076873 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:04:17.076873 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:04:17.076873 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:04:17.076873 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:04:17.077918 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:04:17.077918 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:04:17.077918 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:04:17.077918 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:04:17.077918 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:04:17.077918 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:04:17.077918 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:04:17.077918 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:04:17.077918 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:04:17.077918 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:04:17.077918 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 21:04:17.077918 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:04:17.077918 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 554/1091] 10.222.0.25 () {56 vars in 1120 bytes} [Thu Aug 4 21:04:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:04:17.082996 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1030fc93-325f-490e-b050-bf12eee2dc30 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:17.154273 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107dd557-d6d6-4b51-a98c-fc9bcfaa13f6 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:17.180534 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1030fc93-325f-490e-b050-bf12eee2dc30 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:17.187424 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e174f3a1-859d-4b57-afb0-23de07b655c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:17.188070 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e174f3a1-859d-4b57-afb0-23de07b655c9 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:17.188357 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e174f3a1-859d-4b57-afb0-23de07b655c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:17.188636 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e174f3a1-859d-4b57-afb0-23de07b655c9 None admin] PATH_INFO: `/v3/users/13af542c6bcd45c6b58073c2f623ceba` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:17.189806 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e174f3a1-859d-4b57-afb0-23de07b655c9 None admin] RBAC: Authorizing `identity:get_user(user_id=13af542c6bcd45c6b58073c2f623ceba)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:17.191252 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e174f3a1-859d-4b57-afb0-23de07b655c9 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:17.192892 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 538/1092] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Aug 4 21:04:17 2022] GET /identity/v3/users/13af542c6bcd45c6b58073c2f623ceba => generated 285 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:04:17.198553 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e174f3a1-859d-4b57-afb0-23de07b655c9 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:17.210409 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:17.221413 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c6d3cb-4aea-4f1c-b0b0-08cfa150adfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:17.223040 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c6d3cb-4aea-4f1c-b0b0-08cfa150adfd None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:17.223040 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c6d3cb-4aea-4f1c-b0b0-08cfa150adfd None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:17.223040 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c6d3cb-4aea-4f1c-b0b0-08cfa150adfd None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:17.223550 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c6d3cb-4aea-4f1c-b0b0-08cfa150adfd None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:04:17.223991 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c6d3cb-4aea-4f1c-b0b0-08cfa150adfd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:17.225234 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c6d3cb-4aea-4f1c-b0b0-08cfa150adfd None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:17.234004 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 555/1093] 10.222.0.25 () {56 vars in 1115 bytes} [Thu Aug 4 21:04:17 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:04:17.239777 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c6d3cb-4aea-4f1c-b0b0-08cfa150adfd None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:17.260630 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:17.299823 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e174f3a1-859d-4b57-afb0-23de07b655c9 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:17.333831 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066aac74-3ba8-45ca-b79d-18ef4abe04e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:17.334390 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066aac74-3ba8-45ca-b79d-18ef4abe04e2 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:17.334676 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066aac74-3ba8-45ca-b79d-18ef4abe04e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:17.334955 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066aac74-3ba8-45ca-b79d-18ef4abe04e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:17.337521 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c6d3cb-4aea-4f1c-b0b0-08cfa150adfd None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:17.370478 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f7546c-215f-4a40-9589-6893ea636903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:17.371105 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f7546c-215f-4a40-9589-6893ea636903 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:17.371423 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f7546c-215f-4a40-9589-6893ea636903 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:17.371699 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f7546c-215f-4a40-9589-6893ea636903 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:17.381964 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f7546c-215f-4a40-9589-6893ea636903 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:17.383437 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f7546c-215f-4a40-9589-6893ea636903 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:17.399477 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-c3f7546c-215f-4a40-9589-6893ea636903 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:04:17.399477 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:04:17.399477 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:04:17.399477 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:04:17.399477 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:04:17.399477 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:04:17.399477 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:04:17.399477 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:04:17.399477 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:04:17.399477 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:04:17.399477 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:04:17.399477 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:04:17.399477 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:04:17.399477 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:04:17.399477 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:04:17.400518 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:04:17.400518 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:04:17.400518 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:04:17.400518 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:04:17.400518 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:04:17.400518 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:04:17.400518 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:04:17.400518 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:04:17.400518 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:04:17.400518 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:04:17.400518 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:04:17.400518 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:04:17.400518 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:04:17.400518 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:04:17.400518 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:04:17.400518 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:04:17.400518 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:04:17.401413 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:04:17.401413 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:04:17.401413 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:04:17.401413 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:04:17.401413 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 556/1094] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:04:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:04:17.401703 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066aac74-3ba8-45ca-b79d-18ef4abe04e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:17.403186 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066aac74-3ba8-45ca-b79d-18ef4abe04e2 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:17.406348 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f7546c-215f-4a40-9589-6893ea636903 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:17.447510 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:17.461561 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:17.498691 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 539/1095] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Aug 4 21:04:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 04 21:04:17.502544 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f7546c-215f-4a40-9589-6893ea636903 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:17.504742 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066aac74-3ba8-45ca-b79d-18ef4abe04e2 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:17.537819 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61a7725-15cd-4280-a32d-090338988227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:17.538433 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61a7725-15cd-4280-a32d-090338988227 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:17.538714 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61a7725-15cd-4280-a32d-090338988227 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:17.538985 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61a7725-15cd-4280-a32d-090338988227 None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:17.539698 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61a7725-15cd-4280-a32d-090338988227 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:04:17.540000 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61a7725-15cd-4280-a32d-090338988227 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:17.541363 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61a7725-15cd-4280-a32d-090338988227 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:17.553392 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 557/1096] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:04:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:04:17.558703 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61a7725-15cd-4280-a32d-090338988227 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:17.603840 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066aac74-3ba8-45ca-b79d-18ef4abe04e2 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:17.638231 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9345c46-71de-4ce1-9193-0bc1fec6ce2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:17.638871 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9345c46-71de-4ce1-9193-0bc1fec6ce2b None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:17.639157 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9345c46-71de-4ce1-9193-0bc1fec6ce2b None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:17.639472 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9345c46-71de-4ce1-9193-0bc1fec6ce2b None admin] PATH_INFO: `/v3/projects/878f5e86831941d0be014a94c1315f00` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:17.640627 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9345c46-71de-4ce1-9193-0bc1fec6ce2b None admin] RBAC: Authorizing `identity:get_project(project_id=878f5e86831941d0be014a94c1315f00)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:17.642085 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9345c46-71de-4ce1-9193-0bc1fec6ce2b None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:17.643816 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 540/1097] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Aug 4 21:04:17 2022] GET /identity/v3/projects/878f5e86831941d0be014a94c1315f00 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:04:17.649740 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9345c46-71de-4ce1-9193-0bc1fec6ce2b None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:17.659202 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61a7725-15cd-4280-a32d-090338988227 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:17.661024 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:17.679549 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:17.694503 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5ace8c-3225-491c-8011-e1239d1fb513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:17.695140 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5ace8c-3225-491c-8011-e1239d1fb513 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:17.695517 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5ace8c-3225-491c-8011-e1239d1fb513 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:17.695801 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5ace8c-3225-491c-8011-e1239d1fb513 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:17.702856 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5ace8c-3225-491c-8011-e1239d1fb513 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:17.704328 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5ace8c-3225-491c-8011-e1239d1fb513 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:17.715855 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-2b5ace8c-3225-491c-8011-e1239d1fb513 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 21:04:17.715855 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:04:17.715855 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:04:17.715855 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:04:17.715855 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:04:17.715855 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:04:17.715855 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:04:17.715855 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:04:17.715855 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:04:17.715855 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:04:17.715855 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:04:17.715855 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:04:17.715855 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 21:04:17.715855 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 21:04:17.715855 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 21:04:17.715855 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 21:04:17.716900 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:04:17.716900 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:04:17.716900 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 21:04:17.716900 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:04:17.716900 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 21:04:17.716900 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:04:17.716900 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:04:17.716900 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:04:17.716900 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:04:17.716900 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:04:17.716900 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:04:17.716900 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:04:17.716900 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:04:17.716900 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:04:17.716900 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:04:17.716900 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:04:17.716900 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:04:17.716900 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:04:17.717850 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:04:17.717850 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:04:17.717850 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 21:04:17.717850 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 21:04:17.717850 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 21:04:17.717850 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 21:04:17.717850 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 21:04:17.717850 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 21:04:17.717850 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 21:04:17.717850 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:04:17.717850 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 558/1098] 10.222.0.25 () {56 vars in 1093 bytes} [Thu Aug 4 21:04:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:04:17.722712 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5ace8c-3225-491c-8011-e1239d1fb513 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:17.749273 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9345c46-71de-4ce1-9193-0bc1fec6ce2b None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:17.790023 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cbebc0-d11b-457d-8487-d176b756eb5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:17.791082 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cbebc0-d11b-457d-8487-d176b756eb5d None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:17.791495 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cbebc0-d11b-457d-8487-d176b756eb5d None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:17.791779 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cbebc0-d11b-457d-8487-d176b756eb5d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:17.793208 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cbebc0-d11b-457d-8487-d176b756eb5d None admin] RBAC: Adding query filter params (role.id=a432dd4735944ecfab2fa5ce1705d356, scope.project.id=878f5e86831941d0be014a94c1315f00, user.id=13af542c6bcd45c6b58073c2f623ceba) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:04:17.793782 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cbebc0-d11b-457d-8487-d176b756eb5d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:17.796511 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cbebc0-d11b-457d-8487-d176b756eb5d None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:17.803230 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 541/1099] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Aug 4 21:04:17 2022] GET /identity/v3/role_assignments?user.id=13af542c6bcd45c6b58073c2f623ceba&scope.project.id=878f5e86831941d0be014a94c1315f00&role.id=a432dd4735944ecfab2fa5ce1705d356 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:04:17.830622 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5ace8c-3225-491c-8011-e1239d1fb513 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:17.867298 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1919a9-cbb5-451f-b69c-47dc80e7c539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:17.867884 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1919a9-cbb5-451f-b69c-47dc80e7c539 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:17.868168 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1919a9-cbb5-451f-b69c-47dc80e7c539 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:17.868445 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1919a9-cbb5-451f-b69c-47dc80e7c539 None admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:17.869257 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1919a9-cbb5-451f-b69c-47dc80e7c539 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:04:17.869566 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1919a9-cbb5-451f-b69c-47dc80e7c539 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:17.871034 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1919a9-cbb5-451f-b69c-47dc80e7c539 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:17.891121 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 559/1100] 10.222.0.25 () {56 vars in 1133 bytes} [Thu Aug 4 21:04:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:04:17.897354 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cbebc0-d11b-457d-8487-d176b756eb5d None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:17.993197 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:17.999989 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cbebc0-d11b-457d-8487-d176b756eb5d None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:18.035041 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc7882b-2778-48f6-ae2e-e69042abccc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:18.035700 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc7882b-2778-48f6-ae2e-e69042abccc4 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:18.035987 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc7882b-2778-48f6-ae2e-e69042abccc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:18.036265 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc7882b-2778-48f6-ae2e-e69042abccc4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:18.114461 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc7882b-2778-48f6-ae2e-e69042abccc4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:18.115963 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc7882b-2778-48f6-ae2e-e69042abccc4 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:18.204778 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:18.212184 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 542/1101] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Aug 4 21:04:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 04 21:04:18.217902 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1919a9-cbb5-451f-b69c-47dc80e7c539 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:18.241753 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:18.319562 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1919a9-cbb5-451f-b69c-47dc80e7c539 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:18.352648 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ed4dcb-fcdc-4c5b-a233-5e68e833b772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:18.353298 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ed4dcb-fcdc-4c5b-a233-5e68e833b772 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:18.353583 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ed4dcb-fcdc-4c5b-a233-5e68e833b772 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:18.353859 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ed4dcb-fcdc-4c5b-a233-5e68e833b772 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:18.364356 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ed4dcb-fcdc-4c5b-a233-5e68e833b772 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:18.365828 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ed4dcb-fcdc-4c5b-a233-5e68e833b772 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:18.380825 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-c7ed4dcb-fcdc-4c5b-a233-5e68e833b772 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:04:18.380825 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:04:18.380825 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:04:18.380825 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:04:18.380825 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:04:18.380825 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:04:18.380825 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:04:18.380825 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:04:18.380825 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:04:18.380825 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:04:18.380825 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:04:18.380825 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:04:18.380825 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:04:18.380825 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:04:18.380825 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:04:18.381871 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:04:18.381871 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:04:18.381871 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:04:18.381871 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:04:18.381871 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:04:18.381871 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:04:18.381871 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:04:18.381871 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:04:18.381871 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:04:18.381871 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:04:18.381871 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:04:18.381871 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:04:18.381871 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:04:18.381871 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:04:18.381871 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:04:18.381871 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:04:18.381871 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:04:18.382982 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:04:18.382982 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:04:18.382982 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:04:18.382982 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:04:18.382982 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 560/1102] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:04:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:04:18.389095 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc7882b-2778-48f6-ae2e-e69042abccc4 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:18.486082 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc7882b-2778-48f6-ae2e-e69042abccc4 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:18.518625 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc8eda8-d88e-4079-b78f-c60719f95ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:18.519242 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc8eda8-d88e-4079-b78f-c60719f95ee8 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:18.519551 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc8eda8-d88e-4079-b78f-c60719f95ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:18.519823 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc8eda8-d88e-4079-b78f-c60719f95ee8 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:18.520509 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc8eda8-d88e-4079-b78f-c60719f95ee8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:04:18.520811 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc8eda8-d88e-4079-b78f-c60719f95ee8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:18.522178 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc8eda8-d88e-4079-b78f-c60719f95ee8 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:18.533931 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 543/1103] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:04:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:04:18.539886 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ed4dcb-fcdc-4c5b-a233-5e68e833b772 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:18.607893 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:18.637258 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ed4dcb-fcdc-4c5b-a233-5e68e833b772 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:18.670112 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8d40d1-4691-4d37-b12a-18d6ce6a0576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:18.670746 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8d40d1-4691-4d37-b12a-18d6ce6a0576 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:18.671024 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8d40d1-4691-4d37-b12a-18d6ce6a0576 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:18.671294 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8d40d1-4691-4d37-b12a-18d6ce6a0576 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:18.678255 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8d40d1-4691-4d37-b12a-18d6ce6a0576 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:18.679698 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8d40d1-4691-4d37-b12a-18d6ce6a0576 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:18.691752 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-8e8d40d1-4691-4d37-b12a-18d6ce6a0576 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:04:18.691752 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:04:18.691752 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:04:18.691752 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:04:18.691752 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:04:18.691752 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:04:18.691752 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:04:18.691752 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:04:18.691752 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:04:18.691752 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:04:18.691752 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:04:18.691752 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:04:18.691752 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 21:04:18.691752 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 21:04:18.691752 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 21:04:18.692787 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 21:04:18.692787 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:04:18.692787 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:04:18.692787 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:04:18.692787 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:04:18.692787 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:04:18.692787 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:04:18.692787 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:04:18.692787 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:04:18.692787 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:04:18.692787 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:04:18.692787 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:04:18.692787 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:04:18.692787 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:04:18.692787 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:04:18.692787 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:04:18.692787 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:04:18.693710 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 21:04:18.693710 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 21:04:18.693710 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 21:04:18.693710 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 21:04:18.693710 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 21:04:18.693710 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 21:04:18.693710 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:04:18.693710 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:04:18.693710 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 561/1104] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Aug 4 21:04:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:04:18.698947 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc8eda8-d88e-4079-b78f-c60719f95ee8 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:18.753035 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:18.796757 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc8eda8-d88e-4079-b78f-c60719f95ee8 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:18.829400 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059d3d28-88d2-4e2d-aea7-bebbbe6025b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:18.830015 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059d3d28-88d2-4e2d-aea7-bebbbe6025b9 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:18.830294 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059d3d28-88d2-4e2d-aea7-bebbbe6025b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:18.830563 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059d3d28-88d2-4e2d-aea7-bebbbe6025b9 None admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:18.831304 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059d3d28-88d2-4e2d-aea7-bebbbe6025b9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:04:18.831633 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059d3d28-88d2-4e2d-aea7-bebbbe6025b9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:18.833063 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059d3d28-88d2-4e2d-aea7-bebbbe6025b9 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:18.845772 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 544/1105] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Aug 4 21:04:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:04:18.852126 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8d40d1-4691-4d37-b12a-18d6ce6a0576 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:18.952437 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8d40d1-4691-4d37-b12a-18d6ce6a0576 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:18.981445 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:18.986843 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db045abe-b715-4fc0-bd65-7d3d9758e29d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:18.999677 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db045abe-b715-4fc0-bd65-7d3d9758e29d None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:18.999677 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db045abe-b715-4fc0-bd65-7d3d9758e29d None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:18.999677 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db045abe-b715-4fc0-bd65-7d3d9758e29d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:18.999677 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db045abe-b715-4fc0-bd65-7d3d9758e29d None admin] RBAC: Adding query filter params (role.id=47f3f6a0aef34d25a88aed7f622b2d46, scope.project.id=17dc3d39e2c048ca9070588f5717a578, user.id=6a2ce1df87fa47e5be301916bf3245fb) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:04:18.999677 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db045abe-b715-4fc0-bd65-7d3d9758e29d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:18.999677 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db045abe-b715-4fc0-bd65-7d3d9758e29d None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:19.000118 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 562/1106] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Aug 4 21:04:18 2022] GET /identity/v3/role_assignments?user.id=6a2ce1df87fa47e5be301916bf3245fb&scope.project.id=17dc3d39e2c048ca9070588f5717a578&role.id=47f3f6a0aef34d25a88aed7f622b2d46 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:04:19.850316 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970187d6-16b3-4dee-a289-174e898724f9 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:19.850831 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970187d6-16b3-4dee-a289-174e898724f9 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:19.850923 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970187d6-16b3-4dee-a289-174e898724f9 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:19.852277 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 545/1107] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:04:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:04:19.857437 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b88909-eb5c-4aef-b10c-f99e5e17921f None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:19.857847 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b88909-eb5c-4aef-b10c-f99e5e17921f None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:19.857985 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b88909-eb5c-4aef-b10c-f99e5e17921f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:19.876572 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-b1b88909-eb5c-4aef-b10c-f99e5e17921f None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:04:19.877750 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-b1b88909-eb5c-4aef-b10c-f99e5e17921f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:04:19.943606 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 563/1108] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:04:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 21:04:19.968330 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970187d6-16b3-4dee-a289-174e898724f9 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:20.078420 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970187d6-16b3-4dee-a289-174e898724f9 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:20.092786 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:20.117562 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf5b6b1-3d59-41a5-8064-a725bc1ba641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:20.118180 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf5b6b1-3d59-41a5-8064-a725bc1ba641 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:20.118528 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf5b6b1-3d59-41a5-8064-a725bc1ba641 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:20.118806 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf5b6b1-3d59-41a5-8064-a725bc1ba641 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:20.122014 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf5b6b1-3d59-41a5-8064-a725bc1ba641 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:20.122014 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf5b6b1-3d59-41a5-8064-a725bc1ba641 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:20.123035 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 546/1109] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:04:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:04:20.129489 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b88909-eb5c-4aef-b10c-f99e5e17921f None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:20.269048 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b88909-eb5c-4aef-b10c-f99e5e17921f None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:20.295208 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:20.304430 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805bf7c8-a7ab-41fb-a860-dc2a1ba90576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:20.305080 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805bf7c8-a7ab-41fb-a860-dc2a1ba90576 None admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:20.305344 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805bf7c8-a7ab-41fb-a860-dc2a1ba90576 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:20.305620 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805bf7c8-a7ab-41fb-a860-dc2a1ba90576 None admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:20.306322 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805bf7c8-a7ab-41fb-a860-dc2a1ba90576 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:20.307882 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805bf7c8-a7ab-41fb-a860-dc2a1ba90576 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:20.411438 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 564/1110] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Aug 4 21:04:20 2022] POST /identity/v3/users => generated 286 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:04:22.476215 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c026cbb7-6ae8-4379-aeb0-4f12ae361957 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:22.476993 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c026cbb7-6ae8-4379-aeb0-4f12ae361957 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:22.476993 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c026cbb7-6ae8-4379-aeb0-4f12ae361957 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:22.478475 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 547/1111] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:04:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:04:22.483716 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e473495-4a43-49d4-944a-f45bf7578711 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:22.484118 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e473495-4a43-49d4-944a-f45bf7578711 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:22.484275 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e473495-4a43-49d4-944a-f45bf7578711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:22.502403 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-0e473495-4a43-49d4-944a-f45bf7578711 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:04:22.503618 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-0e473495-4a43-49d4-944a-f45bf7578711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:04:22.568978 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 565/1112] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:04:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:04:22.592504 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c026cbb7-6ae8-4379-aeb0-4f12ae361957 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:22.689243 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c026cbb7-6ae8-4379-aeb0-4f12ae361957 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:22.721802 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52484141-197f-4d40-8748-b039684be23e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:22.722429 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52484141-197f-4d40-8748-b039684be23e None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:22.722712 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52484141-197f-4d40-8748-b039684be23e None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:22.722986 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52484141-197f-4d40-8748-b039684be23e None admin] PATH_INFO: `/v3/roles/70587326d8584316b73b380df2ae8e67` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:22.724430 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52484141-197f-4d40-8748-b039684be23e None admin] RBAC: Authorizing `identity:get_role(role_id=70587326d8584316b73b380df2ae8e67)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:22.725800 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52484141-197f-4d40-8748-b039684be23e None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:22.727181 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 548/1113] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Aug 4 21:04:22 2022] GET /identity/v3/roles/70587326d8584316b73b380df2ae8e67 => generated 223 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:04:22.732713 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e473495-4a43-49d4-944a-f45bf7578711 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:22.828139 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e473495-4a43-49d4-944a-f45bf7578711 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:22.841138 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:22.861763 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d72838b-8737-4a05-806c-9f8d00ae7638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:22.862389 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d72838b-8737-4a05-806c-9f8d00ae7638 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:22.862673 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d72838b-8737-4a05-806c-9f8d00ae7638 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:22.862951 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d72838b-8737-4a05-806c-9f8d00ae7638 None admin] PATH_INFO: `/v3/users/c6a563ccd1d44a599c9fad3081cf8749` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:22.875462 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d72838b-8737-4a05-806c-9f8d00ae7638 None admin] RBAC: Authorizing `identity:get_user(user_id=c6a563ccd1d44a599c9fad3081cf8749)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:22.877519 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d72838b-8737-4a05-806c-9f8d00ae7638 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:22.879506 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 566/1114] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Aug 4 21:04:22 2022] GET /identity/v3/users/c6a563ccd1d44a599c9fad3081cf8749 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:04:22.884961 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52484141-197f-4d40-8748-b039684be23e None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:22.933587 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:22.982903 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52484141-197f-4d40-8748-b039684be23e None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:23.016142 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f582ba2-b29d-48ca-9754-1560a69f3292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:23.016763 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f582ba2-b29d-48ca-9754-1560a69f3292 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:23.017050 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f582ba2-b29d-48ca-9754-1560a69f3292 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:23.017329 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f582ba2-b29d-48ca-9754-1560a69f3292 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:23.082922 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f582ba2-b29d-48ca-9754-1560a69f3292 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:23.084418 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f582ba2-b29d-48ca-9754-1560a69f3292 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:23.130756 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:23.179847 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 549/1115] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Aug 4 21:04:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 21:04:23.186013 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d72838b-8737-4a05-806c-9f8d00ae7638 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:23.321744 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d72838b-8737-4a05-806c-9f8d00ae7638 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:23.363427 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2583284-f8c8-4ec2-9c3d-d41bf1a7de73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:23.364069 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2583284-f8c8-4ec2-9c3d-d41bf1a7de73 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:23.364348 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2583284-f8c8-4ec2-9c3d-d41bf1a7de73 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:23.364621 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2583284-f8c8-4ec2-9c3d-d41bf1a7de73 None admin] PATH_INFO: `/v3/projects/878f5e86831941d0be014a94c1315f00` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:23.365819 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2583284-f8c8-4ec2-9c3d-d41bf1a7de73 None admin] RBAC: Authorizing `identity:get_project(project_id=878f5e86831941d0be014a94c1315f00)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:23.367243 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2583284-f8c8-4ec2-9c3d-d41bf1a7de73 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:23.368942 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 567/1116] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Aug 4 21:04:23 2022] GET /identity/v3/projects/878f5e86831941d0be014a94c1315f00 => generated 311 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:04:23.375162 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f582ba2-b29d-48ca-9754-1560a69f3292 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:23.474220 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f582ba2-b29d-48ca-9754-1560a69f3292 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:23.507338 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ab978c-303d-419d-9cb5-2c3ef7519966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:23.508021 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ab978c-303d-419d-9cb5-2c3ef7519966 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:23.508229 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ab978c-303d-419d-9cb5-2c3ef7519966 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:23.508506 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ab978c-303d-419d-9cb5-2c3ef7519966 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:23.509289 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ab978c-303d-419d-9cb5-2c3ef7519966 None admin] RBAC: Adding query filter params (role.id=70587326d8584316b73b380df2ae8e67, scope.project.id=878f5e86831941d0be014a94c1315f00, user.id=c6a563ccd1d44a599c9fad3081cf8749) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:04:23.509596 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ab978c-303d-419d-9cb5-2c3ef7519966 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:23.511039 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ab978c-303d-419d-9cb5-2c3ef7519966 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:23.517564 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 550/1117] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Aug 4 21:04:23 2022] GET /identity/v3/role_assignments?user.id=c6a563ccd1d44a599c9fad3081cf8749&scope.project.id=878f5e86831941d0be014a94c1315f00&role.id=70587326d8584316b73b380df2ae8e67 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:04:25.586675 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989c56f7-cbc0-4137-a732-c88d1af2850c None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:25.587190 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989c56f7-cbc0-4137-a732-c88d1af2850c None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:25.587272 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989c56f7-cbc0-4137-a732-c88d1af2850c None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:25.589400 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 568/1118] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:04:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:04:25.599361 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b305a51c-a6ae-496f-b1b6-d94e8f217737 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:25.599361 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b305a51c-a6ae-496f-b1b6-d94e8f217737 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:25.599361 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b305a51c-a6ae-496f-b1b6-d94e8f217737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:25.622537 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-b305a51c-a6ae-496f-b1b6-d94e8f217737 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:04:25.624752 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-b305a51c-a6ae-496f-b1b6-d94e8f217737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:04:25.637921 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:25.715020 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 551/1119] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:04:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:04:25.743904 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989c56f7-cbc0-4137-a732-c88d1af2850c None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:25.797024 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:25.858552 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989c56f7-cbc0-4137-a732-c88d1af2850c None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:25.895515 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4afd8c-8b78-41c1-9b40-101cc01a7694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:25.896144 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4afd8c-8b78-41c1-9b40-101cc01a7694 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:25.896422 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4afd8c-8b78-41c1-9b40-101cc01a7694 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:25.896692 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4afd8c-8b78-41c1-9b40-101cc01a7694 None admin] PATH_INFO: `/v3/roles/70587326d8584316b73b380df2ae8e67` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:25.898115 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4afd8c-8b78-41c1-9b40-101cc01a7694 None admin] RBAC: Authorizing `identity:get_role(role_id=70587326d8584316b73b380df2ae8e67)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:25.899512 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4afd8c-8b78-41c1-9b40-101cc01a7694 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:25.900850 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 569/1120] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Aug 4 21:04:25 2022] GET /identity/v3/roles/70587326d8584316b73b380df2ae8e67 => generated 223 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:04:25.906685 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b305a51c-a6ae-496f-b1b6-d94e8f217737 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:26.001884 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b305a51c-a6ae-496f-b1b6-d94e8f217737 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:26.034866 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b587d52c-e9ef-49d0-bcf2-cfcb8dcc864d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:26.035522 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b587d52c-e9ef-49d0-bcf2-cfcb8dcc864d None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:26.035803 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b587d52c-e9ef-49d0-bcf2-cfcb8dcc864d None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:26.036074 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b587d52c-e9ef-49d0-bcf2-cfcb8dcc864d None admin] PATH_INFO: `/v3/users/c6a563ccd1d44a599c9fad3081cf8749` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:26.037217 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b587d52c-e9ef-49d0-bcf2-cfcb8dcc864d None admin] RBAC: Authorizing `identity:get_user(user_id=c6a563ccd1d44a599c9fad3081cf8749)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:26.038649 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b587d52c-e9ef-49d0-bcf2-cfcb8dcc864d None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:26.040274 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 552/1121] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Aug 4 21:04:25 2022] GET /identity/v3/users/c6a563ccd1d44a599c9fad3081cf8749 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:04:26.046070 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4afd8c-8b78-41c1-9b40-101cc01a7694 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:26.170596 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4afd8c-8b78-41c1-9b40-101cc01a7694 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:26.178194 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:26.204588 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fe784c-2849-47f7-8e57-d452a2a7aee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:26.205226 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fe784c-2849-47f7-8e57-d452a2a7aee6 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:26.205511 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fe784c-2849-47f7-8e57-d452a2a7aee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:26.205786 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fe784c-2849-47f7-8e57-d452a2a7aee6 None admin] PATH_INFO: `/v3/projects/878f5e86831941d0be014a94c1315f00` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:26.206932 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fe784c-2849-47f7-8e57-d452a2a7aee6 None admin] RBAC: Authorizing `identity:get_project(project_id=878f5e86831941d0be014a94c1315f00)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:26.208476 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fe784c-2849-47f7-8e57-d452a2a7aee6 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:26.210511 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 570/1122] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Aug 4 21:04:26 2022] GET /identity/v3/projects/878f5e86831941d0be014a94c1315f00 => generated 311 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:04:26.216414 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b587d52c-e9ef-49d0-bcf2-cfcb8dcc864d None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:26.231643 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:26.314300 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b587d52c-e9ef-49d0-bcf2-cfcb8dcc864d None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:26.346681 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffada356-1dae-4b46-8ec4-9ac956ca6160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:26.347361 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffada356-1dae-4b46-8ec4-9ac956ca6160 None admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:26.347641 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffada356-1dae-4b46-8ec4-9ac956ca6160 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:26.347914 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffada356-1dae-4b46-8ec4-9ac956ca6160 None admin] PATH_INFO: `/v3/projects/878f5e86831941d0be014a94c1315f00/users/c6a563ccd1d44a599c9fad3081cf8749/roles/70587326d8584316b73b380df2ae8e67` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:26.350064 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffada356-1dae-4b46-8ec4-9ac956ca6160 None admin] RBAC: Authorizing `identity:create_grant(project_id=878f5e86831941d0be014a94c1315f00, user_id=c6a563ccd1d44a599c9fad3081cf8749, role_id=70587326d8584316b73b380df2ae8e67)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:26.351520 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffada356-1dae-4b46-8ec4-9ac956ca6160 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:26.429319 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 553/1123] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Aug 4 21:04:26 2022] PUT /identity/v3/projects/878f5e86831941d0be014a94c1315f00/users/c6a563ccd1d44a599c9fad3081cf8749/roles/70587326d8584316b73b380df2ae8e67 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:04:28.486215 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f899b343-0b7d-4f5d-85e9-91db4f70916a None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:28.486759 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f899b343-0b7d-4f5d-85e9-91db4f70916a None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:28.486759 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f899b343-0b7d-4f5d-85e9-91db4f70916a None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:28.488159 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 571/1124] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:04:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:04:28.493294 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24497ac-1115-47a1-83a4-89d70272bfc7 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:28.493701 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24497ac-1115-47a1-83a4-89d70272bfc7 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:28.493841 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24497ac-1115-47a1-83a4-89d70272bfc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:28.512057 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-f24497ac-1115-47a1-83a4-89d70272bfc7 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:04:28.513221 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-f24497ac-1115-47a1-83a4-89d70272bfc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:04:28.576426 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 554/1125] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:04:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:04:28.599669 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f899b343-0b7d-4f5d-85e9-91db4f70916a None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:28.696187 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f899b343-0b7d-4f5d-85e9-91db4f70916a None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:28.728181 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6539bde-95f9-4730-b5c5-69b483c91be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:28.728803 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6539bde-95f9-4730-b5c5-69b483c91be0 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:28.729080 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6539bde-95f9-4730-b5c5-69b483c91be0 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:28.729356 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6539bde-95f9-4730-b5c5-69b483c91be0 None admin] PATH_INFO: `/v3/roles/70587326d8584316b73b380df2ae8e67` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:28.730756 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6539bde-95f9-4730-b5c5-69b483c91be0 None admin] RBAC: Authorizing `identity:get_role(role_id=70587326d8584316b73b380df2ae8e67)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:28.732147 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6539bde-95f9-4730-b5c5-69b483c91be0 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:28.733500 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 572/1126] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Aug 4 21:04:28 2022] GET /identity/v3/roles/70587326d8584316b73b380df2ae8e67 => generated 223 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:04:28.739292 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24497ac-1115-47a1-83a4-89d70272bfc7 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:28.741409 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:28.836656 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24497ac-1115-47a1-83a4-89d70272bfc7 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:28.868644 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9e031d-fc57-43d3-b8cb-c14f75feac27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:28.869275 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e031d-fc57-43d3-b8cb-c14f75feac27 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:28.869555 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e031d-fc57-43d3-b8cb-c14f75feac27 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:28.869829 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e031d-fc57-43d3-b8cb-c14f75feac27 None admin] PATH_INFO: `/v3/users/c6a563ccd1d44a599c9fad3081cf8749` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:28.870971 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9e031d-fc57-43d3-b8cb-c14f75feac27 None admin] RBAC: Authorizing `identity:get_user(user_id=c6a563ccd1d44a599c9fad3081cf8749)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:28.872441 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9e031d-fc57-43d3-b8cb-c14f75feac27 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:28.874039 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 555/1127] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Aug 4 21:04:28 2022] GET /identity/v3/users/c6a563ccd1d44a599c9fad3081cf8749 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:04:28.880483 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6539bde-95f9-4730-b5c5-69b483c91be0 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:28.923987 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:28.983411 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6539bde-95f9-4730-b5c5-69b483c91be0 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:29.016169 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08887e91-1819-4147-b14b-42d75af8f13f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:29.016723 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08887e91-1819-4147-b14b-42d75af8f13f None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:29.017006 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08887e91-1819-4147-b14b-42d75af8f13f None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:29.017282 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08887e91-1819-4147-b14b-42d75af8f13f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:29.081480 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08887e91-1819-4147-b14b-42d75af8f13f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:29.082915 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08887e91-1819-4147-b14b-42d75af8f13f None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:29.122394 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:29.177762 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 573/1128] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Aug 4 21:04:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 21:04:29.183866 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9e031d-fc57-43d3-b8cb-c14f75feac27 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:29.253732 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:29.283695 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9e031d-fc57-43d3-b8cb-c14f75feac27 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:29.316050 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28bfe57-8064-45d6-9a83-ed5818440583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:29.316680 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28bfe57-8064-45d6-9a83-ed5818440583 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:29.316962 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28bfe57-8064-45d6-9a83-ed5818440583 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:29.317236 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28bfe57-8064-45d6-9a83-ed5818440583 None admin] PATH_INFO: `/v3/projects/878f5e86831941d0be014a94c1315f00` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:29.318380 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28bfe57-8064-45d6-9a83-ed5818440583 None admin] RBAC: Authorizing `identity:get_project(project_id=878f5e86831941d0be014a94c1315f00)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:29.319853 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28bfe57-8064-45d6-9a83-ed5818440583 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:29.321510 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 556/1129] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Aug 4 21:04:29 2022] GET /identity/v3/projects/878f5e86831941d0be014a94c1315f00 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:04:29.327656 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08887e91-1819-4147-b14b-42d75af8f13f None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:29.423706 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08887e91-1819-4147-b14b-42d75af8f13f None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:29.455693 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c062d0e-da65-455f-93bd-e36f7b3507b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:29.456301 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c062d0e-da65-455f-93bd-e36f7b3507b3 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:29.456585 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c062d0e-da65-455f-93bd-e36f7b3507b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:29.456860 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c062d0e-da65-455f-93bd-e36f7b3507b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:29.457634 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c062d0e-da65-455f-93bd-e36f7b3507b3 None admin] RBAC: Adding query filter params (role.id=70587326d8584316b73b380df2ae8e67, scope.project.id=878f5e86831941d0be014a94c1315f00, user.id=c6a563ccd1d44a599c9fad3081cf8749) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:04:29.457942 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c062d0e-da65-455f-93bd-e36f7b3507b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:29.459382 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c062d0e-da65-455f-93bd-e36f7b3507b3 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:29.464939 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 574/1130] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Aug 4 21:04:29 2022] GET /identity/v3/role_assignments?user.id=c6a563ccd1d44a599c9fad3081cf8749&scope.project.id=878f5e86831941d0be014a94c1315f00&role.id=70587326d8584316b73b380df2ae8e67 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:04:31.472903 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8770b0b1-0a75-4be7-a17c-e6e502566fdb None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:31.473421 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8770b0b1-0a75-4be7-a17c-e6e502566fdb None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:31.473527 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8770b0b1-0a75-4be7-a17c-e6e502566fdb None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:31.474811 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 557/1131] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:04:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:04:31.480063 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1fff0e-e1bd-4a93-aaf7-ac055b85dffe None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:31.480462 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1fff0e-e1bd-4a93-aaf7-ac055b85dffe None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:31.480563 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1fff0e-e1bd-4a93-aaf7-ac055b85dffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:31.493816 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:31.498924 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-8f1fff0e-e1bd-4a93-aaf7-ac055b85dffe None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:04:31.500168 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-8f1fff0e-e1bd-4a93-aaf7-ac055b85dffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:04:31.575022 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 575/1132] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:04:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:04:31.597923 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8770b0b1-0a75-4be7-a17c-e6e502566fdb None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:31.695458 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8770b0b1-0a75-4be7-a17c-e6e502566fdb None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:31.727337 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c543966b-b367-4c7e-b392-6a3645a2cfa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:31.727975 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c543966b-b367-4c7e-b392-6a3645a2cfa1 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:31.728253 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c543966b-b367-4c7e-b392-6a3645a2cfa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:31.728525 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c543966b-b367-4c7e-b392-6a3645a2cfa1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:31.729640 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c543966b-b367-4c7e-b392-6a3645a2cfa1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:31.731132 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c543966b-b367-4c7e-b392-6a3645a2cfa1 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:31.732669 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 558/1133] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:04:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:04:31.739097 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1fff0e-e1bd-4a93-aaf7-ac055b85dffe None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:31.837524 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1fff0e-e1bd-4a93-aaf7-ac055b85dffe None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:31.872493 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d55540-3bd9-4fca-a699-3a7e8e2f1ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:31.873253 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d55540-3bd9-4fca-a699-3a7e8e2f1ada None admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:31.873660 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d55540-3bd9-4fca-a699-3a7e8e2f1ada None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:31.874043 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d55540-3bd9-4fca-a699-3a7e8e2f1ada None admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:31.874727 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d55540-3bd9-4fca-a699-3a7e8e2f1ada None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:31.876261 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d55540-3bd9-4fca-a699-3a7e8e2f1ada None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:31.881385 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:31.989534 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 576/1134] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Aug 4 21:04:31 2022] POST /identity/v3/projects => generated 311 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:04:34.070746 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0021b78-e5a2-4b75-8773-3a5d3653de2b None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:34.071304 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0021b78-e5a2-4b75-8773-3a5d3653de2b None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:34.071304 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0021b78-e5a2-4b75-8773-3a5d3653de2b None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:34.072684 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 559/1135] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:04:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:04:34.077917 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a60cbc9-efef-400c-ad1d-efe9ca8524a6 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:34.078350 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a60cbc9-efef-400c-ad1d-efe9ca8524a6 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:34.078488 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a60cbc9-efef-400c-ad1d-efe9ca8524a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:34.097177 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-7a60cbc9-efef-400c-ad1d-efe9ca8524a6 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:04:34.098469 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-7a60cbc9-efef-400c-ad1d-efe9ca8524a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:04:34.167224 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 577/1136] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:04:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:04:34.192341 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0021b78-e5a2-4b75-8773-3a5d3653de2b None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:34.201839 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:34.303067 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0021b78-e5a2-4b75-8773-3a5d3653de2b None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:34.342759 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79520e34-8162-4bdc-8813-b746131263f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:34.343310 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79520e34-8162-4bdc-8813-b746131263f4 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:34.343617 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79520e34-8162-4bdc-8813-b746131263f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:34.343888 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79520e34-8162-4bdc-8813-b746131263f4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:34.345036 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79520e34-8162-4bdc-8813-b746131263f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:34.347007 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79520e34-8162-4bdc-8813-b746131263f4 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:34.348650 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 560/1137] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:04:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:04:34.354443 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a60cbc9-efef-400c-ad1d-efe9ca8524a6 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:34.456860 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a60cbc9-efef-400c-ad1d-efe9ca8524a6 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:34.474355 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:34.492451 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd183738-8fa9-4905-b808-b5777ddb50fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:34.493078 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd183738-8fa9-4905-b808-b5777ddb50fd None admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:34.493362 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd183738-8fa9-4905-b808-b5777ddb50fd None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:34.493640 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd183738-8fa9-4905-b808-b5777ddb50fd None admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:34.494340 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd183738-8fa9-4905-b808-b5777ddb50fd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:34.495955 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd183738-8fa9-4905-b808-b5777ddb50fd None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:34.639648 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 578/1138] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Aug 4 21:04:34 2022] POST /identity/v3/users => generated 286 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:04:36.793609 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d587473-8f19-434f-b974-53335a4c2bd3 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:36.794169 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d587473-8f19-434f-b974-53335a4c2bd3 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:36.794169 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d587473-8f19-434f-b974-53335a4c2bd3 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:36.795560 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 561/1139] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:04:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:04:36.802747 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7508f255-7ef9-4ec0-bf46-6de66d06d345 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:36.803614 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7508f255-7ef9-4ec0-bf46-6de66d06d345 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:36.803614 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7508f255-7ef9-4ec0-bf46-6de66d06d345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:36.822866 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-7508f255-7ef9-4ec0-bf46-6de66d06d345 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:04:36.824095 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-7508f255-7ef9-4ec0-bf46-6de66d06d345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:04:36.896787 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 579/1140] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:04:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:04:36.926239 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d587473-8f19-434f-b974-53335a4c2bd3 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:37.005473 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:37.032494 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d587473-8f19-434f-b974-53335a4c2bd3 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:37.068304 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec9bc05-fc08-437f-8fda-a60c2539c557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:37.068864 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec9bc05-fc08-437f-8fda-a60c2539c557 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:37.069147 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec9bc05-fc08-437f-8fda-a60c2539c557 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:37.069420 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec9bc05-fc08-437f-8fda-a60c2539c557 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:37.080478 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec9bc05-fc08-437f-8fda-a60c2539c557 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:04:37.085360 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-4ec9bc05-fc08-437f-8fda-a60c2539c557 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:04:37.085360 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:04:37.085360 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:04:37.085360 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:04:37.085360 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:04:37.085360 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:04:37.085360 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:04:37.085360 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:04:37.085360 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:04:37.085360 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:04:37.085360 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:04:37.085360 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:04:37.085360 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:04:37.085360 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:04:37.085360 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:04:37.085360 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:04:37.086802 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:04:37.086802 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:04:37.086802 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:04:37.086802 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:04:37.086802 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:04:37.086802 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:04:37.086802 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:04:37.086802 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:04:37.086802 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:04:37.086802 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:04:37.086802 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:04:37.086802 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:04:37.086802 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:04:37.086802 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:04:37.086802 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:04:37.086802 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:04:37.088676 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:04:37.088676 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:04:37.088676 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:04:37.088676 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:04:37.088676 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:04:37.088676 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:04:37.088676 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:04:37.088676 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:04:37.088676 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:04:37.088676 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:04:37.088676 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:04:37.088676 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:04:37.088676 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 562/1141] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Aug 4 21:04:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:04:37.092626 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7508f255-7ef9-4ec0-bf46-6de66d06d345 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:37.199277 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7508f255-7ef9-4ec0-bf46-6de66d06d345 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:37.203339 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:37.251113 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f418dd7-18a8-4903-9d16-f3b183e8e05d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:37.254672 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f418dd7-18a8-4903-9d16-f3b183e8e05d None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:37.254672 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f418dd7-18a8-4903-9d16-f3b183e8e05d None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:37.254672 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f418dd7-18a8-4903-9d16-f3b183e8e05d None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:37.254672 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f418dd7-18a8-4903-9d16-f3b183e8e05d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:04:37.254672 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f418dd7-18a8-4903-9d16-f3b183e8e05d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:37.254672 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f418dd7-18a8-4903-9d16-f3b183e8e05d None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:37.262909 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 580/1142] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Aug 4 21:04:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:04:37.268780 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec9bc05-fc08-437f-8fda-a60c2539c557 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:37.385698 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec9bc05-fc08-437f-8fda-a60c2539c557 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:37.421613 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb807ec-ce37-4d67-afb0-70ae4bf21070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:37.422775 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb807ec-ce37-4d67-afb0-70ae4bf21070 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:37.423057 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb807ec-ce37-4d67-afb0-70ae4bf21070 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:37.423349 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb807ec-ce37-4d67-afb0-70ae4bf21070 None admin] PATH_INFO: `/v3/users/408a8482b15e41cbb4e58684bbdd4d9d` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:37.425203 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:37.437502 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb807ec-ce37-4d67-afb0-70ae4bf21070 None admin] RBAC: Authorizing `identity:get_user(user_id=408a8482b15e41cbb4e58684bbdd4d9d)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:37.443427 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb807ec-ce37-4d67-afb0-70ae4bf21070 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:37.443427 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 563/1143] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Aug 4 21:04:37 2022] GET /identity/v3/users/408a8482b15e41cbb4e58684bbdd4d9d => generated 286 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:04:37.447034 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f418dd7-18a8-4903-9d16-f3b183e8e05d None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:37.551130 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f418dd7-18a8-4903-9d16-f3b183e8e05d None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:37.584193 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da5c1c1-fb80-4725-a850-9198e1a42d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:37.584803 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da5c1c1-fb80-4725-a850-9198e1a42d3c None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:37.585080 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da5c1c1-fb80-4725-a850-9198e1a42d3c None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:37.585349 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da5c1c1-fb80-4725-a850-9198e1a42d3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:37.621487 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:37.661037 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da5c1c1-fb80-4725-a850-9198e1a42d3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:37.662473 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da5c1c1-fb80-4725-a850-9198e1a42d3c None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:37.763915 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 581/1144] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Aug 4 21:04:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 21:04:37.772277 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb807ec-ce37-4d67-afb0-70ae4bf21070 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:37.871655 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb807ec-ce37-4d67-afb0-70ae4bf21070 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:37.904451 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04474453-de12-4721-94b1-d8fb3c85d44e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:37.905095 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04474453-de12-4721-94b1-d8fb3c85d44e None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:37.905357 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04474453-de12-4721-94b1-d8fb3c85d44e None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:37.905626 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04474453-de12-4721-94b1-d8fb3c85d44e None admin] PATH_INFO: `/v3/projects/a6cd4da5e69c47f9bb2a7e6d761b2dc8` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:37.907358 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04474453-de12-4721-94b1-d8fb3c85d44e None admin] RBAC: Authorizing `identity:get_project(project_id=a6cd4da5e69c47f9bb2a7e6d761b2dc8)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:37.908793 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04474453-de12-4721-94b1-d8fb3c85d44e None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:37.910702 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 564/1145] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Aug 4 21:04:37 2022] GET /identity/v3/projects/a6cd4da5e69c47f9bb2a7e6d761b2dc8 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:04:37.917435 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da5c1c1-fb80-4725-a850-9198e1a42d3c None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:37.985388 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:38.019646 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da5c1c1-fb80-4725-a850-9198e1a42d3c None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:38.063394 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956099e3-92d5-43f4-a8ef-d418b7767f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:38.064167 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956099e3-92d5-43f4-a8ef-d418b7767f3a None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:38.064462 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956099e3-92d5-43f4-a8ef-d418b7767f3a None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:38.064737 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956099e3-92d5-43f4-a8ef-d418b7767f3a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:38.065850 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956099e3-92d5-43f4-a8ef-d418b7767f3a None admin] RBAC: Adding query filter params (role.id=a432dd4735944ecfab2fa5ce1705d356, scope.project.id=a6cd4da5e69c47f9bb2a7e6d761b2dc8, user.id=408a8482b15e41cbb4e58684bbdd4d9d) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:04:38.066467 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956099e3-92d5-43f4-a8ef-d418b7767f3a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:38.067974 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956099e3-92d5-43f4-a8ef-d418b7767f3a None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:38.074349 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 582/1146] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Aug 4 21:04:37 2022] GET /identity/v3/role_assignments?user.id=408a8482b15e41cbb4e58684bbdd4d9d&scope.project.id=a6cd4da5e69c47f9bb2a7e6d761b2dc8&role.id=a432dd4735944ecfab2fa5ce1705d356 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:04:40.104852 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19709095-0836-4431-8c95-31b8e828ddba None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:40.105407 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19709095-0836-4431-8c95-31b8e828ddba None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:40.105407 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19709095-0836-4431-8c95-31b8e828ddba None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:40.106742 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 565/1147] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:04:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:04:40.112632 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a89e768-6300-45b7-91a0-775bd062dcbb None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:40.113054 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a89e768-6300-45b7-91a0-775bd062dcbb None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:40.113185 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a89e768-6300-45b7-91a0-775bd062dcbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:40.130769 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-4a89e768-6300-45b7-91a0-775bd062dcbb None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:04:40.131971 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-4a89e768-6300-45b7-91a0-775bd062dcbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:04:40.195981 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 583/1148] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:04:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:04:40.217850 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19709095-0836-4431-8c95-31b8e828ddba None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:40.277857 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:40.317591 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19709095-0836-4431-8c95-31b8e828ddba None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:40.350597 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce223aa6-4f53-4126-b5a3-66f5e890b806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:40.351231 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce223aa6-4f53-4126-b5a3-66f5e890b806 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:40.351542 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce223aa6-4f53-4126-b5a3-66f5e890b806 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:40.351819 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce223aa6-4f53-4126-b5a3-66f5e890b806 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:40.362173 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce223aa6-4f53-4126-b5a3-66f5e890b806 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:04:40.366973 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-ce223aa6-4f53-4126-b5a3-66f5e890b806 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:04:40.366973 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:04:40.366973 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:04:40.366973 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:04:40.366973 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:04:40.366973 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:04:40.366973 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:04:40.366973 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:04:40.366973 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:04:40.366973 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:04:40.366973 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:04:40.366973 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:04:40.366973 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:04:40.366973 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:04:40.366973 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:04:40.366973 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:04:40.368082 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:04:40.368082 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:04:40.368082 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:04:40.368082 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:04:40.368082 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:04:40.368082 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:04:40.368082 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:04:40.368082 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:04:40.368082 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:04:40.368082 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:04:40.368082 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:04:40.368082 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:04:40.368082 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:04:40.368082 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:04:40.368082 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:04:40.368082 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:04:40.369011 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:04:40.369011 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:04:40.369011 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:04:40.369011 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:04:40.369011 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:04:40.369011 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:04:40.369011 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:04:40.369011 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:04:40.369011 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:04:40.369011 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:04:40.369011 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:04:40.369011 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:04:40.369011 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 566/1149] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Aug 4 21:04:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:04:40.374388 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a89e768-6300-45b7-91a0-775bd062dcbb None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:40.431216 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:40.471310 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a89e768-6300-45b7-91a0-775bd062dcbb None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:40.503461 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34229e0-3c86-4ce3-a19c-08b60f66fb17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:40.504068 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34229e0-3c86-4ce3-a19c-08b60f66fb17 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:40.504350 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34229e0-3c86-4ce3-a19c-08b60f66fb17 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:40.504625 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34229e0-3c86-4ce3-a19c-08b60f66fb17 None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:40.505331 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34229e0-3c86-4ce3-a19c-08b60f66fb17 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:04:40.505633 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34229e0-3c86-4ce3-a19c-08b60f66fb17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:40.506990 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34229e0-3c86-4ce3-a19c-08b60f66fb17 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:40.515142 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 584/1150] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Aug 4 21:04:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:04:40.521654 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce223aa6-4f53-4126-b5a3-66f5e890b806 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:40.616320 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce223aa6-4f53-4126-b5a3-66f5e890b806 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:40.635089 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:40.650646 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2ffe65-a9f1-42c2-ae59-236d1018e7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:40.651289 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2ffe65-a9f1-42c2-ae59-236d1018e7f8 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:40.651604 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2ffe65-a9f1-42c2-ae59-236d1018e7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:40.651909 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2ffe65-a9f1-42c2-ae59-236d1018e7f8 None admin] PATH_INFO: `/v3/users/408a8482b15e41cbb4e58684bbdd4d9d` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:40.653081 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2ffe65-a9f1-42c2-ae59-236d1018e7f8 None admin] RBAC: Authorizing `identity:get_user(user_id=408a8482b15e41cbb4e58684bbdd4d9d)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:40.654518 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2ffe65-a9f1-42c2-ae59-236d1018e7f8 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:40.656145 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 567/1151] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Aug 4 21:04:40 2022] GET /identity/v3/users/408a8482b15e41cbb4e58684bbdd4d9d => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:04:40.662377 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34229e0-3c86-4ce3-a19c-08b60f66fb17 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:40.762477 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34229e0-3c86-4ce3-a19c-08b60f66fb17 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:40.784323 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:40.795493 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cffebc-9210-41df-9982-076eff620d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:40.796312 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cffebc-9210-41df-9982-076eff620d3f None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:40.796592 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cffebc-9210-41df-9982-076eff620d3f None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:40.796859 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cffebc-9210-41df-9982-076eff620d3f None admin] PATH_INFO: `/v3/projects/a6cd4da5e69c47f9bb2a7e6d761b2dc8` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:40.797992 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cffebc-9210-41df-9982-076eff620d3f None admin] RBAC: Authorizing `identity:get_project(project_id=a6cd4da5e69c47f9bb2a7e6d761b2dc8)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:40.799435 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cffebc-9210-41df-9982-076eff620d3f None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:40.801072 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 585/1152] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Aug 4 21:04:40 2022] GET /identity/v3/projects/a6cd4da5e69c47f9bb2a7e6d761b2dc8 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:04:40.806973 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2ffe65-a9f1-42c2-ae59-236d1018e7f8 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:40.903509 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2ffe65-a9f1-42c2-ae59-236d1018e7f8 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:40.935391 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d58ef39-e601-4911-9944-7e24b64c3b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:40.936044 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d58ef39-e601-4911-9944-7e24b64c3b7c None admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:40.936324 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d58ef39-e601-4911-9944-7e24b64c3b7c None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:40.936592 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d58ef39-e601-4911-9944-7e24b64c3b7c None admin] PATH_INFO: `/v3/projects/a6cd4da5e69c47f9bb2a7e6d761b2dc8/users/408a8482b15e41cbb4e58684bbdd4d9d/roles/a432dd4735944ecfab2fa5ce1705d356` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:40.938549 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d58ef39-e601-4911-9944-7e24b64c3b7c None admin] RBAC: Authorizing `identity:create_grant(project_id=a6cd4da5e69c47f9bb2a7e6d761b2dc8, user_id=408a8482b15e41cbb4e58684bbdd4d9d, role_id=a432dd4735944ecfab2fa5ce1705d356)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:40.940001 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d58ef39-e601-4911-9944-7e24b64c3b7c None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:41.021051 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 568/1153] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Aug 4 21:04:40 2022] PUT /identity/v3/projects/a6cd4da5e69c47f9bb2a7e6d761b2dc8/users/408a8482b15e41cbb4e58684bbdd4d9d/roles/a432dd4735944ecfab2fa5ce1705d356 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:04:43.014597 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ec58f-8d80-4ac3-8714-d4a7d3a4ae4c None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:43.015128 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ec58f-8d80-4ac3-8714-d4a7d3a4ae4c None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:43.015128 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ec58f-8d80-4ac3-8714-d4a7d3a4ae4c None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:43.016571 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 586/1154] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:04:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:04:43.021921 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523cd075-1854-494f-ae6b-7c5d71cf9565 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:43.022323 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523cd075-1854-494f-ae6b-7c5d71cf9565 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:43.025621 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523cd075-1854-494f-ae6b-7c5d71cf9565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:43.041766 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-523cd075-1854-494f-ae6b-7c5d71cf9565 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:04:43.042956 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-523cd075-1854-494f-ae6b-7c5d71cf9565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:04:43.085011 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:43.107589 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 569/1155] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:04:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:04:43.131413 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202ec58f-8d80-4ac3-8714-d4a7d3a4ae4c None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:43.234769 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202ec58f-8d80-4ac3-8714-d4a7d3a4ae4c None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:43.268557 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9e5cb2-bc8a-4cf6-b669-9051aa6a15c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:43.269127 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9e5cb2-bc8a-4cf6-b669-9051aa6a15c6 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:43.269411 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9e5cb2-bc8a-4cf6-b669-9051aa6a15c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:43.269685 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9e5cb2-bc8a-4cf6-b669-9051aa6a15c6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:43.280159 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9e5cb2-bc8a-4cf6-b669-9051aa6a15c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:04:43.286053 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-9c9e5cb2-bc8a-4cf6-b669-9051aa6a15c6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:04:43.286053 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:04:43.286053 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:04:43.286053 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:04:43.286053 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:04:43.286053 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:04:43.286053 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:04:43.286053 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:04:43.286053 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:04:43.286053 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:04:43.286053 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:04:43.286053 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:04:43.286053 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:04:43.286053 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:04:43.286053 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:04:43.286053 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:04:43.287173 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:04:43.287173 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:04:43.287173 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:04:43.287173 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:04:43.287173 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:04:43.287173 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:04:43.287173 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:04:43.287173 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:04:43.287173 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:04:43.287173 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:04:43.287173 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:04:43.287173 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:04:43.287173 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:04:43.287173 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:04:43.287173 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:04:43.287173 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:04:43.288276 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:04:43.288276 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:04:43.288276 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:04:43.288276 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:04:43.288276 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:04:43.288276 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:04:43.288276 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:04:43.288276 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:04:43.288276 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:04:43.288276 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:04:43.288276 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:04:43.288276 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:04:43.288276 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 587/1156] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Aug 4 21:04:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:04:43.293327 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523cd075-1854-494f-ae6b-7c5d71cf9565 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:43.392759 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523cd075-1854-494f-ae6b-7c5d71cf9565 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:43.426563 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9de18ec-abb7-49c6-a04e-5287c9534e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:43.427183 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9de18ec-abb7-49c6-a04e-5287c9534e0b None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:43.427508 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9de18ec-abb7-49c6-a04e-5287c9534e0b None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:43.427786 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9de18ec-abb7-49c6-a04e-5287c9534e0b None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:43.428498 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9de18ec-abb7-49c6-a04e-5287c9534e0b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:04:43.428805 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9de18ec-abb7-49c6-a04e-5287c9534e0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:43.430193 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9de18ec-abb7-49c6-a04e-5287c9534e0b None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:43.438384 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 570/1157] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Aug 4 21:04:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:04:43.444315 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9e5cb2-bc8a-4cf6-b669-9051aa6a15c6 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:43.495686 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:43.546031 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9e5cb2-bc8a-4cf6-b669-9051aa6a15c6 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:43.579977 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c8816b-8f07-4ef6-8d7d-2f71b7eec944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:43.580643 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8816b-8f07-4ef6-8d7d-2f71b7eec944 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:43.580928 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8816b-8f07-4ef6-8d7d-2f71b7eec944 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:43.581204 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8816b-8f07-4ef6-8d7d-2f71b7eec944 None admin] PATH_INFO: `/v3/users/408a8482b15e41cbb4e58684bbdd4d9d` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:43.582380 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c8816b-8f07-4ef6-8d7d-2f71b7eec944 None admin] RBAC: Authorizing `identity:get_user(user_id=408a8482b15e41cbb4e58684bbdd4d9d)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:43.583878 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c8816b-8f07-4ef6-8d7d-2f71b7eec944 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:43.585500 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 588/1158] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Aug 4 21:04:43 2022] GET /identity/v3/users/408a8482b15e41cbb4e58684bbdd4d9d => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:04:43.591363 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9de18ec-abb7-49c6-a04e-5287c9534e0b None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:43.633191 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:43.693729 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9de18ec-abb7-49c6-a04e-5287c9534e0b None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:43.727566 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e252e3a-6a72-4eb4-a0fd-3504fc2a695e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:43.728188 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e252e3a-6a72-4eb4-a0fd-3504fc2a695e None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:43.728476 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e252e3a-6a72-4eb4-a0fd-3504fc2a695e None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:43.728753 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e252e3a-6a72-4eb4-a0fd-3504fc2a695e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:43.797755 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e252e3a-6a72-4eb4-a0fd-3504fc2a695e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:43.799262 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e252e3a-6a72-4eb4-a0fd-3504fc2a695e None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:43.840174 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:43.900710 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 571/1159] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Aug 4 21:04:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 21:04:43.906754 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c8816b-8f07-4ef6-8d7d-2f71b7eec944 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:44.011882 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c8816b-8f07-4ef6-8d7d-2f71b7eec944 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:44.029560 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:44.046385 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e61a8c8-c714-4fbf-b3d5-cd77858ac5f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:44.047565 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e61a8c8-c714-4fbf-b3d5-cd77858ac5f0 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:44.047901 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e61a8c8-c714-4fbf-b3d5-cd77858ac5f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:44.048123 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e61a8c8-c714-4fbf-b3d5-cd77858ac5f0 None admin] PATH_INFO: `/v3/projects/a6cd4da5e69c47f9bb2a7e6d761b2dc8` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:44.049276 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e61a8c8-c714-4fbf-b3d5-cd77858ac5f0 None admin] RBAC: Authorizing `identity:get_project(project_id=a6cd4da5e69c47f9bb2a7e6d761b2dc8)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:44.050708 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e61a8c8-c714-4fbf-b3d5-cd77858ac5f0 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:44.052421 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 589/1160] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Aug 4 21:04:43 2022] GET /identity/v3/projects/a6cd4da5e69c47f9bb2a7e6d761b2dc8 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:04:44.058547 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e252e3a-6a72-4eb4-a0fd-3504fc2a695e None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:44.164554 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e252e3a-6a72-4eb4-a0fd-3504fc2a695e None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:44.200722 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e6b419-91d7-4b6a-9892-959d9646b866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:44.201347 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e6b419-91d7-4b6a-9892-959d9646b866 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:44.201633 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e6b419-91d7-4b6a-9892-959d9646b866 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:44.201911 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e6b419-91d7-4b6a-9892-959d9646b866 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:44.202703 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e6b419-91d7-4b6a-9892-959d9646b866 None admin] RBAC: Adding query filter params (role.id=a432dd4735944ecfab2fa5ce1705d356, scope.project.id=a6cd4da5e69c47f9bb2a7e6d761b2dc8, user.id=408a8482b15e41cbb4e58684bbdd4d9d) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:04:44.203020 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e6b419-91d7-4b6a-9892-959d9646b866 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:44.204540 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e6b419-91d7-4b6a-9892-959d9646b866 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:44.210853 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 572/1161] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Aug 4 21:04:44 2022] GET /identity/v3/role_assignments?user.id=408a8482b15e41cbb4e58684bbdd4d9d&scope.project.id=a6cd4da5e69c47f9bb2a7e6d761b2dc8&role.id=a432dd4735944ecfab2fa5ce1705d356 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:04:46.232399 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2582aee-80c6-490f-aea8-9d32d19afd8d None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:46.232794 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2582aee-80c6-490f-aea8-9d32d19afd8d None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:46.232956 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2582aee-80c6-490f-aea8-9d32d19afd8d None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:46.234320 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 590/1162] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:04:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:04:46.239443 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c0b78c-60eb-4287-8933-671deb57074a None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:46.239863 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c0b78c-60eb-4287-8933-671deb57074a None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:46.240008 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c0b78c-60eb-4287-8933-671deb57074a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:46.247351 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:46.257733 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-91c0b78c-60eb-4287-8933-671deb57074a None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:04:46.259111 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-91c0b78c-60eb-4287-8933-671deb57074a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:04:46.325418 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 573/1163] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:04:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:04:46.348432 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2582aee-80c6-490f-aea8-9d32d19afd8d None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:46.451229 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2582aee-80c6-490f-aea8-9d32d19afd8d None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:46.486890 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1fee93-48e3-43d1-8641-7b531e97f5cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:46.487525 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1fee93-48e3-43d1-8641-7b531e97f5cb None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:46.487812 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1fee93-48e3-43d1-8641-7b531e97f5cb None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:46.488087 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1fee93-48e3-43d1-8641-7b531e97f5cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:46.544391 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:46.557247 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1fee93-48e3-43d1-8641-7b531e97f5cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:46.558709 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1fee93-48e3-43d1-8641-7b531e97f5cb None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:46.655994 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 591/1164] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Aug 4 21:04:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 21:04:46.662311 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c0b78c-60eb-4287-8933-671deb57074a None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:46.767216 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c0b78c-60eb-4287-8933-671deb57074a None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:46.799829 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce035b5-a71f-43be-84b6-35d2a0cf0c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:46.800457 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce035b5-a71f-43be-84b6-35d2a0cf0c00 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:46.800739 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce035b5-a71f-43be-84b6-35d2a0cf0c00 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:46.801013 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce035b5-a71f-43be-84b6-35d2a0cf0c00 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:46.802824 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:46.807971 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce035b5-a71f-43be-84b6-35d2a0cf0c00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:46.809419 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce035b5-a71f-43be-84b6-35d2a0cf0c00 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:46.821373 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-dce035b5-a71f-43be-84b6-35d2a0cf0c00 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 04 21:04:46.821373 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:04:46.821373 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 04 21:04:46.821373 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 04 21:04:46.821373 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 21:04:46.821373 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 21:04:46.821373 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 21:04:46.821373 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 21:04:46.821373 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 04 21:04:46.821373 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:04:46.821373 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 21:04:46.821373 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:04:46.821373 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:04:46.821373 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:04:46.821373 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:04:46.821373 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:04:46.821373 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:04:46.822625 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:04:46.822625 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:04:46.822625 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:04:46.822625 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:04:46.822625 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:04:46.822625 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:04:46.822625 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:04:46.822625 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:04:46.822625 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:04:46.822625 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:04:46.822625 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:04:46.822625 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:04:46.822625 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:04:46.822625 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:04:46.822625 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:04:46.822625 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:04:46.823704 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:04:46.823704 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:04:46.823704 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:04:46.823704 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:04:46.823704 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:04:46.823704 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:04:46.823704 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:04:46.823704 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:04:46.823704 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:04:46.823704 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:04:46.823704 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 04 21:04:46.823704 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:04:46.823704 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 04 21:04:46.823704 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:04:46.823704 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 574/1165] 10.222.0.25 () {56 vars in 1117 bytes} [Thu Aug 4 21:04:46 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:04:46.829392 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1fee93-48e3-43d1-8641-7b531e97f5cb None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:46.909404 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:46.923865 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1fee93-48e3-43d1-8641-7b531e97f5cb None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:46.957681 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bf3a5d-4f5b-4b04-8cab-4eff95078b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:46.958282 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf3a5d-4f5b-4b04-8cab-4eff95078b0d None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:46.958559 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf3a5d-4f5b-4b04-8cab-4eff95078b0d None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:46.958826 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf3a5d-4f5b-4b04-8cab-4eff95078b0d None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:46.959554 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bf3a5d-4f5b-4b04-8cab-4eff95078b0d None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:04:46.959854 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bf3a5d-4f5b-4b04-8cab-4eff95078b0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:46.961215 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bf3a5d-4f5b-4b04-8cab-4eff95078b0d None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:46.968017 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 592/1166] 10.222.0.25 () {56 vars in 1115 bytes} [Thu Aug 4 21:04:46 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:04:48.980105 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c911e769-9b30-42c2-b6d7-fe25333fe58e None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:48.980673 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c911e769-9b30-42c2-b6d7-fe25333fe58e None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:48.980673 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c911e769-9b30-42c2-b6d7-fe25333fe58e None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:48.982013 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 575/1167] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:04:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:04:48.987078 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cede8d-8aaa-4940-8e37-5f8ba36e063f None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:48.987517 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cede8d-8aaa-4940-8e37-5f8ba36e063f None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:48.987655 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cede8d-8aaa-4940-8e37-5f8ba36e063f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:49.005648 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-b7cede8d-8aaa-4940-8e37-5f8ba36e063f None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:04:49.006809 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-b7cede8d-8aaa-4940-8e37-5f8ba36e063f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:04:49.070584 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 593/1168] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:04:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:04:49.093670 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c911e769-9b30-42c2-b6d7-fe25333fe58e None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:49.191038 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c911e769-9b30-42c2-b6d7-fe25333fe58e None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:49.222937 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0765dc-907f-4ea7-943f-188b4417a1c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:49.223586 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0765dc-907f-4ea7-943f-188b4417a1c4 None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:49.223894 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0765dc-907f-4ea7-943f-188b4417a1c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:49.224166 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0765dc-907f-4ea7-943f-188b4417a1c4 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:49.225331 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0765dc-907f-4ea7-943f-188b4417a1c4 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:49.226231 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0765dc-907f-4ea7-943f-188b4417a1c4 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:49.292550 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 576/1169] 10.222.0.25 () {60 vars in 1138 bytes} [Thu Aug 4 21:04:49 2022] POST /identity/v3/domains => generated 267 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:04:51.373830 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048e3e86-33ba-4d6c-9d9f-9f6298499655 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:51.374364 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048e3e86-33ba-4d6c-9d9f-9f6298499655 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:51.374364 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048e3e86-33ba-4d6c-9d9f-9f6298499655 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:51.375756 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 594/1170] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:04:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:04:51.381407 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719d97fc-c731-42a1-a52e-f1eb99afa2d4 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:51.381927 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719d97fc-c731-42a1-a52e-f1eb99afa2d4 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:51.382050 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719d97fc-c731-42a1-a52e-f1eb99afa2d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:51.400920 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-719d97fc-c731-42a1-a52e-f1eb99afa2d4 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:04:51.402122 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-719d97fc-c731-42a1-a52e-f1eb99afa2d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:04:51.465448 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:51.484953 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 577/1171] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:04:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:04:51.506486 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048e3e86-33ba-4d6c-9d9f-9f6298499655 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:51.588932 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:51.609510 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048e3e86-33ba-4d6c-9d9f-9f6298499655 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:51.643803 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899226d6-2357-4c58-9af9-421e314fbeb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:51.644534 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899226d6-2357-4c58-9af9-421e314fbeb4 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:51.644885 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899226d6-2357-4c58-9af9-421e314fbeb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:51.645213 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899226d6-2357-4c58-9af9-421e314fbeb4 None admin] PATH_INFO: `/v3/domains/42c7819adce34e78b95f717cc85a5ab5` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:51.657120 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899226d6-2357-4c58-9af9-421e314fbeb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=42c7819adce34e78b95f717cc85a5ab5)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:51.658628 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899226d6-2357-4c58-9af9-421e314fbeb4 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:51.660656 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 595/1172] 10.222.0.25 () {56 vars in 1183 bytes} [Thu Aug 4 21:04:51 2022] GET /identity/v3/domains/42c7819adce34e78b95f717cc85a5ab5 => generated 267 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:04:51.667466 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719d97fc-c731-42a1-a52e-f1eb99afa2d4 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:51.765316 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719d97fc-c731-42a1-a52e-f1eb99afa2d4 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:51.798287 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ebc641-d710-4144-8672-2164fb124054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:51.798902 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ebc641-d710-4144-8672-2164fb124054 None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:51.799414 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ebc641-d710-4144-8672-2164fb124054 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:51.799694 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ebc641-d710-4144-8672-2164fb124054 None admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:51.800377 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ebc641-d710-4144-8672-2164fb124054 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:51.801812 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ebc641-d710-4144-8672-2164fb124054 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:51.883520 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 578/1173] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Aug 4 21:04:51 2022] POST /identity/v3/projects => generated 361 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:04:53.921253 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939cded7-5f4e-4e0c-b456-f8fdf8688f2e None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:53.922323 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939cded7-5f4e-4e0c-b456-f8fdf8688f2e None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:53.922323 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939cded7-5f4e-4e0c-b456-f8fdf8688f2e None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:53.923736 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 596/1174] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:04:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:04:53.928962 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca56b67-14d4-44d9-b2ce-414f2f6e2d9a None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:53.929406 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca56b67-14d4-44d9-b2ce-414f2f6e2d9a None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:53.929514 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca56b67-14d4-44d9-b2ce-414f2f6e2d9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:53.947690 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-9ca56b67-14d4-44d9-b2ce-414f2f6e2d9a None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:04:53.948879 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-9ca56b67-14d4-44d9-b2ce-414f2f6e2d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:04:53.976604 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:54.015444 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 579/1175] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:04:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:04:54.040535 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939cded7-5f4e-4e0c-b456-f8fdf8688f2e None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:54.154385 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939cded7-5f4e-4e0c-b456-f8fdf8688f2e None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:54.190135 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ad6dce-09bd-4464-9b71-ae8580efa5b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:54.190688 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ad6dce-09bd-4464-9b71-ae8580efa5b5 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:54.190966 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ad6dce-09bd-4464-9b71-ae8580efa5b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:54.191238 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ad6dce-09bd-4464-9b71-ae8580efa5b5 None admin] PATH_INFO: `/v3/domains/42c7819adce34e78b95f717cc85a5ab5` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:54.192421 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ad6dce-09bd-4464-9b71-ae8580efa5b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=42c7819adce34e78b95f717cc85a5ab5)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:54.193864 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ad6dce-09bd-4464-9b71-ae8580efa5b5 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:54.195509 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 597/1176] 10.222.0.25 () {56 vars in 1183 bytes} [Thu Aug 4 21:04:54 2022] GET /identity/v3/domains/42c7819adce34e78b95f717cc85a5ab5 => generated 267 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:04:54.201776 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca56b67-14d4-44d9-b2ce-414f2f6e2d9a None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:54.319343 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca56b67-14d4-44d9-b2ce-414f2f6e2d9a None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:54.358797 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cab2a57-3fdc-45cc-b6a1-37171d9ec5dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:54.360006 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab2a57-3fdc-45cc-b6a1-37171d9ec5dd None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:54.360289 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab2a57-3fdc-45cc-b6a1-37171d9ec5dd None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:54.360559 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab2a57-3fdc-45cc-b6a1-37171d9ec5dd None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:54.361260 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cab2a57-3fdc-45cc-b6a1-37171d9ec5dd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:54.362713 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cab2a57-3fdc-45cc-b6a1-37171d9ec5dd None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:54.468524 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 580/1177] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Aug 4 21:04:54 2022] POST /identity/v3/users => generated 311 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:04:56.574379 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb28f3a3-baa4-4062-bfdf-64a4d9e96923 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:56.574910 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb28f3a3-baa4-4062-bfdf-64a4d9e96923 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:56.574910 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb28f3a3-baa4-4062-bfdf-64a4d9e96923 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:56.576236 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 598/1178] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:04:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:04:56.582327 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff48376-74a5-43ae-b83b-f2237cf2204e None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:56.582810 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff48376-74a5-43ae-b83b-f2237cf2204e None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:56.583019 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff48376-74a5-43ae-b83b-f2237cf2204e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:56.601957 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-0ff48376-74a5-43ae-b83b-f2237cf2204e None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:04:56.603146 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-0ff48376-74a5-43ae-b83b-f2237cf2204e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:04:56.620728 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:56.672659 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 581/1179] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:04:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 21:04:56.696641 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb28f3a3-baa4-4062-bfdf-64a4d9e96923 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:56.707875 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:56.805433 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb28f3a3-baa4-4062-bfdf-64a4d9e96923 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:56.839696 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485c25f9-c247-42ef-bbd8-10ff6825b90f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:56.840259 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485c25f9-c247-42ef-bbd8-10ff6825b90f None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:56.840539 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485c25f9-c247-42ef-bbd8-10ff6825b90f None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:56.840807 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485c25f9-c247-42ef-bbd8-10ff6825b90f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:56.850913 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485c25f9-c247-42ef-bbd8-10ff6825b90f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:04:56.855776 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-485c25f9-c247-42ef-bbd8-10ff6825b90f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:04:56.855776 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:04:56.855776 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:04:56.855776 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:04:56.855776 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:04:56.855776 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:04:56.855776 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:04:56.855776 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:04:56.855776 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:04:56.855776 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:04:56.855776 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:04:56.855776 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:04:56.855776 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:04:56.855776 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:04:56.855776 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:04:56.855776 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:04:56.856864 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:04:56.856864 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:04:56.856864 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:04:56.856864 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:04:56.856864 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:04:56.856864 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:04:56.856864 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:04:56.856864 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:04:56.856864 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:04:56.856864 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:04:56.856864 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:04:56.856864 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:04:56.856864 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:04:56.856864 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:04:56.856864 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:04:56.856864 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:04:56.857927 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:04:56.857927 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:04:56.857927 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:04:56.857927 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:04:56.857927 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:04:56.857927 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:04:56.857927 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:04:56.857927 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:04:56.857927 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:04:56.857927 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:04:56.857927 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:04:56.857927 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:04:56.857927 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 599/1180] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Aug 4 21:04:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:04:56.864028 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff48376-74a5-43ae-b83b-f2237cf2204e None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:56.964827 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff48376-74a5-43ae-b83b-f2237cf2204e None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:56.999071 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44b5681-7ef7-41cc-b100-529d01eccbd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:56.999765 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44b5681-7ef7-41cc-b100-529d01eccbd3 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:57.000652 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44b5681-7ef7-41cc-b100-529d01eccbd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:57.000952 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44b5681-7ef7-41cc-b100-529d01eccbd3 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:57.001730 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44b5681-7ef7-41cc-b100-529d01eccbd3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:04:57.002071 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44b5681-7ef7-41cc-b100-529d01eccbd3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:57.003544 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44b5681-7ef7-41cc-b100-529d01eccbd3 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:57.012353 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 582/1181] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Aug 4 21:04:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:04:57.018213 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485c25f9-c247-42ef-bbd8-10ff6825b90f None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:57.089646 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:57.124819 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485c25f9-c247-42ef-bbd8-10ff6825b90f None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:57.163699 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75a539d-fed6-457d-bf32-fc71b529d37f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:57.164343 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75a539d-fed6-457d-bf32-fc71b529d37f None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:57.164627 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75a539d-fed6-457d-bf32-fc71b529d37f None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:57.164906 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75a539d-fed6-457d-bf32-fc71b529d37f None admin] PATH_INFO: `/v3/users/856558e9aa2046748507df44f611a8bf` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:57.179746 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75a539d-fed6-457d-bf32-fc71b529d37f None admin] RBAC: Authorizing `identity:get_user(user_id=856558e9aa2046748507df44f611a8bf)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:57.181366 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75a539d-fed6-457d-bf32-fc71b529d37f None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:57.183110 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 600/1182] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Aug 4 21:04:57 2022] GET /identity/v3/users/856558e9aa2046748507df44f611a8bf => generated 311 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:04:57.189225 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44b5681-7ef7-41cc-b100-529d01eccbd3 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:57.207091 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:57.297661 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44b5681-7ef7-41cc-b100-529d01eccbd3 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:57.330836 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159b3c63-4e0b-466f-97d1-d421942a382c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:57.331490 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159b3c63-4e0b-466f-97d1-d421942a382c None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:57.331770 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159b3c63-4e0b-466f-97d1-d421942a382c None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:57.332040 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159b3c63-4e0b-466f-97d1-d421942a382c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:57.397772 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159b3c63-4e0b-466f-97d1-d421942a382c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:57.399154 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159b3c63-4e0b-466f-97d1-d421942a382c None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:57.413948 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:57.495800 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 583/1183] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Aug 4 21:04:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 21:04:57.501977 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75a539d-fed6-457d-bf32-fc71b529d37f None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:57.608586 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75a539d-fed6-457d-bf32-fc71b529d37f None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:57.639505 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:57.649045 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9940ac81-f857-4008-8baf-7b3ddd0a6d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:57.649636 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9940ac81-f857-4008-8baf-7b3ddd0a6d07 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:57.649923 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9940ac81-f857-4008-8baf-7b3ddd0a6d07 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:57.650200 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9940ac81-f857-4008-8baf-7b3ddd0a6d07 None admin] PATH_INFO: `/v3/projects/2cfd3fe44eb14fe98155c99ee3189c04` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:57.651393 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9940ac81-f857-4008-8baf-7b3ddd0a6d07 None admin] RBAC: Authorizing `identity:get_project(project_id=2cfd3fe44eb14fe98155c99ee3189c04)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:57.652835 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9940ac81-f857-4008-8baf-7b3ddd0a6d07 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:57.654515 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 601/1184] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Aug 4 21:04:57 2022] GET /identity/v3/projects/2cfd3fe44eb14fe98155c99ee3189c04 => generated 361 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:04:57.660904 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159b3c63-4e0b-466f-97d1-d421942a382c None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:57.766419 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159b3c63-4e0b-466f-97d1-d421942a382c None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:57.779261 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:57.799929 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc53db8b-456d-4ad0-b309-d2dd94db7b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:57.800538 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc53db8b-456d-4ad0-b309-d2dd94db7b72 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:57.800821 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc53db8b-456d-4ad0-b309-d2dd94db7b72 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:57.801096 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc53db8b-456d-4ad0-b309-d2dd94db7b72 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:57.801872 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc53db8b-456d-4ad0-b309-d2dd94db7b72 None admin] RBAC: Adding query filter params (role.id=a432dd4735944ecfab2fa5ce1705d356, scope.project.id=2cfd3fe44eb14fe98155c99ee3189c04, user.id=856558e9aa2046748507df44f611a8bf) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:04:57.802182 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc53db8b-456d-4ad0-b309-d2dd94db7b72 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:57.803662 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc53db8b-456d-4ad0-b309-d2dd94db7b72 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:57.809081 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 584/1185] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Aug 4 21:04:57 2022] GET /identity/v3/role_assignments?user.id=856558e9aa2046748507df44f611a8bf&scope.project.id=2cfd3fe44eb14fe98155c99ee3189c04&role.id=a432dd4735944ecfab2fa5ce1705d356 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:04:59.793131 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3a5c8b-e726-4647-89f3-270bc6f59ea0 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:59.793725 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3a5c8b-e726-4647-89f3-270bc6f59ea0 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:59.793725 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3a5c8b-e726-4647-89f3-270bc6f59ea0 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:59.795053 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 602/1186] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:04:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:04:59.800461 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef035e7-6360-4402-9242-0c101ef04a04 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:59.800922 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef035e7-6360-4402-9242-0c101ef04a04 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:59.801020 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef035e7-6360-4402-9242-0c101ef04a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:59.818888 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-6ef035e7-6360-4402-9242-0c101ef04a04 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:04:59.820105 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-6ef035e7-6360-4402-9242-0c101ef04a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:04:59.885471 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 585/1187] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:04:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:04:59.909624 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3a5c8b-e726-4647-89f3-270bc6f59ea0 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:00.014946 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3a5c8b-e726-4647-89f3-270bc6f59ea0 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:00.048919 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04000c22-ecbc-41f6-8625-f456af8c5720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:00.049485 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04000c22-ecbc-41f6-8625-f456af8c5720 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:00.049768 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04000c22-ecbc-41f6-8625-f456af8c5720 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:00.050042 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04000c22-ecbc-41f6-8625-f456af8c5720 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:00.060521 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04000c22-ecbc-41f6-8625-f456af8c5720 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:05:00.067879 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-04000c22-ecbc-41f6-8625-f456af8c5720 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:05:00.067879 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:05:00.067879 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:05:00.067879 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:05:00.067879 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:05:00.067879 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:05:00.067879 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:05:00.067879 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:05:00.067879 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:05:00.067879 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:05:00.067879 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:05:00.067879 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:05:00.067879 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:05:00.067879 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:05:00.067879 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:05:00.067879 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:05:00.069048 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:05:00.069048 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:05:00.069048 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:05:00.069048 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:05:00.069048 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:05:00.069048 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:05:00.069048 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:05:00.069048 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:05:00.069048 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:05:00.069048 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:05:00.069048 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:05:00.069048 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:05:00.069048 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:05:00.069048 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:05:00.069048 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:05:00.069048 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:05:00.070293 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:05:00.070293 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:05:00.070293 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:05:00.070293 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:05:00.070293 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:05:00.070293 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:05:00.070293 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:05:00.070293 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:05:00.070293 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:05:00.070293 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:05:00.070293 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:05:00.070293 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:05:00.070293 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 603/1188] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Aug 4 21:04:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:05:00.075105 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef035e7-6360-4402-9242-0c101ef04a04 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:00.173067 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef035e7-6360-4402-9242-0c101ef04a04 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:00.182311 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:05:00.208303 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c44575-3b9f-4454-83a5-4d518030cec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:00.208899 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c44575-3b9f-4454-83a5-4d518030cec6 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:00.209188 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c44575-3b9f-4454-83a5-4d518030cec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:00.209463 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c44575-3b9f-4454-83a5-4d518030cec6 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:00.210173 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c44575-3b9f-4454-83a5-4d518030cec6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:05:00.210477 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c44575-3b9f-4454-83a5-4d518030cec6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:00.211923 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c44575-3b9f-4454-83a5-4d518030cec6 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:00.220464 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 586/1189] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Aug 4 21:05:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:05:00.226186 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04000c22-ecbc-41f6-8625-f456af8c5720 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:00.289722 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:05:00.343723 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04000c22-ecbc-41f6-8625-f456af8c5720 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:00.377178 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd442d95-7547-4297-ae00-4764d6f1bb0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:00.377814 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd442d95-7547-4297-ae00-4764d6f1bb0d None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:00.378093 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd442d95-7547-4297-ae00-4764d6f1bb0d None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:00.378379 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd442d95-7547-4297-ae00-4764d6f1bb0d None admin] PATH_INFO: `/v3/users/856558e9aa2046748507df44f611a8bf` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:00.379625 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd442d95-7547-4297-ae00-4764d6f1bb0d None admin] RBAC: Authorizing `identity:get_user(user_id=856558e9aa2046748507df44f611a8bf)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:00.381055 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd442d95-7547-4297-ae00-4764d6f1bb0d None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:00.382656 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 604/1190] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Aug 4 21:05:00 2022] GET /identity/v3/users/856558e9aa2046748507df44f611a8bf => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:05:00.388623 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c44575-3b9f-4454-83a5-4d518030cec6 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:00.487225 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c44575-3b9f-4454-83a5-4d518030cec6 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:00.521139 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a61c2b-8dfd-4767-96fb-4a8626465213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:00.521778 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a61c2b-8dfd-4767-96fb-4a8626465213 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:00.522060 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a61c2b-8dfd-4767-96fb-4a8626465213 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:00.522334 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a61c2b-8dfd-4767-96fb-4a8626465213 None admin] PATH_INFO: `/v3/projects/2cfd3fe44eb14fe98155c99ee3189c04` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:00.523507 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a61c2b-8dfd-4767-96fb-4a8626465213 None admin] RBAC: Authorizing `identity:get_project(project_id=2cfd3fe44eb14fe98155c99ee3189c04)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:00.524962 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a61c2b-8dfd-4767-96fb-4a8626465213 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:00.526640 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 587/1191] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Aug 4 21:05:00 2022] GET /identity/v3/projects/2cfd3fe44eb14fe98155c99ee3189c04 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:05:00.533315 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd442d95-7547-4297-ae00-4764d6f1bb0d None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:00.634163 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd442d95-7547-4297-ae00-4764d6f1bb0d None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:00.657447 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:05:00.669663 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae57e64-a1f0-4925-bca3-91bd629764dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:00.670403 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae57e64-a1f0-4925-bca3-91bd629764dd None admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:00.670690 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae57e64-a1f0-4925-bca3-91bd629764dd None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:00.670972 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae57e64-a1f0-4925-bca3-91bd629764dd None admin] PATH_INFO: `/v3/projects/2cfd3fe44eb14fe98155c99ee3189c04/users/856558e9aa2046748507df44f611a8bf/roles/a432dd4735944ecfab2fa5ce1705d356` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:00.673537 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae57e64-a1f0-4925-bca3-91bd629764dd None admin] RBAC: Authorizing `identity:create_grant(project_id=2cfd3fe44eb14fe98155c99ee3189c04, user_id=856558e9aa2046748507df44f611a8bf, role_id=a432dd4735944ecfab2fa5ce1705d356)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:00.674989 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae57e64-a1f0-4925-bca3-91bd629764dd None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:00.742045 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 605/1192] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Aug 4 21:05:00 2022] PUT /identity/v3/projects/2cfd3fe44eb14fe98155c99ee3189c04/users/856558e9aa2046748507df44f611a8bf/roles/a432dd4735944ecfab2fa5ce1705d356 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:05:02.749166 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eafa188-eac4-465c-9dfb-51fb90d09822 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:02.749791 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eafa188-eac4-465c-9dfb-51fb90d09822 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:02.749791 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eafa188-eac4-465c-9dfb-51fb90d09822 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:02.751096 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 588/1193] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:05:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:05:02.756332 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf714b1-25f0-48ee-abec-cb034b5b50d1 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:02.756786 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf714b1-25f0-48ee-abec-cb034b5b50d1 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:02.756982 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf714b1-25f0-48ee-abec-cb034b5b50d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:02.775820 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-5bf714b1-25f0-48ee-abec-cb034b5b50d1 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:05:02.777019 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-5bf714b1-25f0-48ee-abec-cb034b5b50d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:05:02.845180 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 606/1194] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:05:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:05:02.868705 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eafa188-eac4-465c-9dfb-51fb90d09822 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:02.917005 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:05:02.978734 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eafa188-eac4-465c-9dfb-51fb90d09822 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:03.013225 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99e7717-19c9-499f-a6ac-043b5bdaec37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:03.013835 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99e7717-19c9-499f-a6ac-043b5bdaec37 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:03.014115 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99e7717-19c9-499f-a6ac-043b5bdaec37 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:03.014385 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99e7717-19c9-499f-a6ac-043b5bdaec37 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:03.025621 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99e7717-19c9-499f-a6ac-043b5bdaec37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:05:03.031606 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-e99e7717-19c9-499f-a6ac-043b5bdaec37 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:05:03.031606 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:05:03.031606 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:05:03.031606 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:05:03.031606 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:05:03.031606 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:05:03.031606 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:05:03.031606 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:05:03.031606 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:05:03.031606 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:05:03.031606 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:05:03.031606 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:05:03.031606 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:05:03.031606 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:05:03.031606 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:05:03.031606 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:05:03.033232 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:05:03.033232 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:05:03.033232 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:05:03.033232 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:05:03.033232 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:05:03.033232 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:05:03.033232 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:05:03.033232 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:05:03.033232 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:05:03.033232 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:05:03.033232 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:05:03.033232 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:05:03.033232 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:05:03.033232 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:05:03.033232 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:05:03.033232 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:05:03.034142 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:05:03.034142 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:05:03.034142 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:05:03.034142 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:05:03.034142 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:05:03.034142 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:05:03.034142 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:05:03.034142 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:05:03.034142 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:05:03.034142 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:05:03.034142 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:05:03.034142 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:05:03.034142 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 589/1195] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Aug 4 21:05:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:05:03.039470 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf714b1-25f0-48ee-abec-cb034b5b50d1 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:03.138801 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf714b1-25f0-48ee-abec-cb034b5b50d1 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:03.152807 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:05:03.174679 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046df855-f342-4dfd-ae1b-bde57b2427ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:03.175377 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046df855-f342-4dfd-ae1b-bde57b2427ed None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:03.175661 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046df855-f342-4dfd-ae1b-bde57b2427ed None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:03.175934 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046df855-f342-4dfd-ae1b-bde57b2427ed None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:03.176656 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046df855-f342-4dfd-ae1b-bde57b2427ed None admin] RBAC: Adding query filter params (name=admin) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:05:03.176956 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046df855-f342-4dfd-ae1b-bde57b2427ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:03.178335 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046df855-f342-4dfd-ae1b-bde57b2427ed None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:03.189058 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 607/1196] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Aug 4 21:05:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:05:03.195050 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99e7717-19c9-499f-a6ac-043b5bdaec37 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:03.312510 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:05:03.316805 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99e7717-19c9-499f-a6ac-043b5bdaec37 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:03.359469 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b17d40f-6814-4e7c-91c5-5117e0007955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:03.360120 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b17d40f-6814-4e7c-91c5-5117e0007955 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:03.360401 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b17d40f-6814-4e7c-91c5-5117e0007955 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:03.360678 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b17d40f-6814-4e7c-91c5-5117e0007955 None admin] PATH_INFO: `/v3/users/856558e9aa2046748507df44f611a8bf` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:03.362426 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b17d40f-6814-4e7c-91c5-5117e0007955 None admin] RBAC: Authorizing `identity:get_user(user_id=856558e9aa2046748507df44f611a8bf)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:03.363915 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b17d40f-6814-4e7c-91c5-5117e0007955 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:03.365559 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 590/1197] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Aug 4 21:05:03 2022] GET /identity/v3/users/856558e9aa2046748507df44f611a8bf => generated 311 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:05:03.371578 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046df855-f342-4dfd-ae1b-bde57b2427ed None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:03.500547 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046df855-f342-4dfd-ae1b-bde57b2427ed None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:03.542722 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1757135d-7d00-49ea-99e0-db2af510a290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:03.543360 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1757135d-7d00-49ea-99e0-db2af510a290 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:03.543643 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1757135d-7d00-49ea-99e0-db2af510a290 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:03.543914 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1757135d-7d00-49ea-99e0-db2af510a290 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:03.587902 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:05:03.617175 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1757135d-7d00-49ea-99e0-db2af510a290 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:03.618685 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1757135d-7d00-49ea-99e0-db2af510a290 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:03.745366 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 608/1198] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Aug 4 21:05:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 21:05:03.751988 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b17d40f-6814-4e7c-91c5-5117e0007955 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:03.885658 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b17d40f-6814-4e7c-91c5-5117e0007955 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:03.924064 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6eea87-44f2-4009-8ec8-661138b64606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:03.924711 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6eea87-44f2-4009-8ec8-661138b64606 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:03.924991 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6eea87-44f2-4009-8ec8-661138b64606 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:03.925269 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6eea87-44f2-4009-8ec8-661138b64606 None admin] PATH_INFO: `/v3/projects/2cfd3fe44eb14fe98155c99ee3189c04` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:03.926445 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6eea87-44f2-4009-8ec8-661138b64606 None admin] RBAC: Authorizing `identity:get_project(project_id=2cfd3fe44eb14fe98155c99ee3189c04)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:03.927937 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6eea87-44f2-4009-8ec8-661138b64606 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:03.929629 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 591/1199] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Aug 4 21:05:03 2022] GET /identity/v3/projects/2cfd3fe44eb14fe98155c99ee3189c04 => generated 361 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:05:03.936236 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1757135d-7d00-49ea-99e0-db2af510a290 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:04.020615 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:05:04.048187 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1757135d-7d00-49ea-99e0-db2af510a290 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:04.082980 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19490784-ba83-492e-ae3f-a571159a0c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:04.083628 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19490784-ba83-492e-ae3f-a571159a0c8d None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:04.083909 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19490784-ba83-492e-ae3f-a571159a0c8d None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:04.084181 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19490784-ba83-492e-ae3f-a571159a0c8d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:04.084967 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19490784-ba83-492e-ae3f-a571159a0c8d None admin] RBAC: Adding query filter params (role.id=a432dd4735944ecfab2fa5ce1705d356, scope.project.id=2cfd3fe44eb14fe98155c99ee3189c04, user.id=856558e9aa2046748507df44f611a8bf) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:05:04.085272 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19490784-ba83-492e-ae3f-a571159a0c8d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:04.086717 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19490784-ba83-492e-ae3f-a571159a0c8d None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:04.093207 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 609/1200] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Aug 4 21:05:03 2022] GET /identity/v3/role_assignments?user.id=856558e9aa2046748507df44f611a8bf&scope.project.id=2cfd3fe44eb14fe98155c99ee3189c04&role.id=a432dd4735944ecfab2fa5ce1705d356 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:05:13.918490 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=83 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:05:14.090192 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=31 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:19.123491 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65acfd59-eb90-42ed-82af-2dd2abbfc8d5 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:19.124142 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65acfd59-eb90-42ed-82af-2dd2abbfc8d5 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:19.124142 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65acfd59-eb90-42ed-82af-2dd2abbfc8d5 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:19.125405 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 592/1201] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:06:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:06:19.130670 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d330cfb-790d-4a9b-b7bc-439898ffe19e None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:19.131098 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d330cfb-790d-4a9b-b7bc-439898ffe19e None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:19.131252 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d330cfb-790d-4a9b-b7bc-439898ffe19e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:19.149937 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-2d330cfb-790d-4a9b-b7bc-439898ffe19e None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:06:19.152307 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-2d330cfb-790d-4a9b-b7bc-439898ffe19e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:19.223528 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 610/1202] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:06:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 21:06:19.247799 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65acfd59-eb90-42ed-82af-2dd2abbfc8d5 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:19.353138 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65acfd59-eb90-42ed-82af-2dd2abbfc8d5 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:19.387862 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883da16d-b4fb-478b-b440-d4898ea7aeac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:19.388428 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883da16d-b4fb-478b-b440-d4898ea7aeac None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:19.388716 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883da16d-b4fb-478b-b440-d4898ea7aeac None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:19.388989 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883da16d-b4fb-478b-b440-d4898ea7aeac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:19.402147 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883da16d-b4fb-478b-b440-d4898ea7aeac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:19.403653 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883da16d-b4fb-478b-b440-d4898ea7aeac None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:19.420909 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-883da16d-b4fb-478b-b440-d4898ea7aeac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:06:19.420909 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:06:19.420909 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:06:19.420909 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:06:19.420909 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:06:19.420909 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:06:19.420909 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:06:19.420909 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:06:19.420909 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:06:19.420909 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:06:19.420909 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:06:19.420909 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:06:19.420909 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:06:19.420909 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:06:19.420909 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:06:19.426059 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:06:19.426059 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:06:19.426059 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:06:19.426059 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:06:19.426059 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:06:19.426059 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:06:19.426059 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:06:19.426059 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:06:19.426059 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:06:19.426059 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:06:19.426059 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:06:19.426059 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:06:19.426059 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:06:19.426059 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:06:19.426059 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:06:19.426059 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:06:19.426059 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:06:19.427261 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:06:19.427261 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:06:19.427261 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:06:19.427261 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:06:19.427261 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 593/1203] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:06:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:06:19.429585 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d330cfb-790d-4a9b-b7bc-439898ffe19e None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:19.536991 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d330cfb-790d-4a9b-b7bc-439898ffe19e None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:19.568698 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:19.574643 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a806df3d-52e1-4def-b7b4-4d9e55de1fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:19.575273 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a806df3d-52e1-4def-b7b4-4d9e55de1fbe None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:19.575594 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a806df3d-52e1-4def-b7b4-4d9e55de1fbe None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:19.575873 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a806df3d-52e1-4def-b7b4-4d9e55de1fbe None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:19.576572 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a806df3d-52e1-4def-b7b4-4d9e55de1fbe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:06:19.576882 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a806df3d-52e1-4def-b7b4-4d9e55de1fbe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:19.578251 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a806df3d-52e1-4def-b7b4-4d9e55de1fbe None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:19.590352 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 611/1204] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:06:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:06:19.596108 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883da16d-b4fb-478b-b440-d4898ea7aeac None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:19.651914 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:19.700604 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883da16d-b4fb-478b-b440-d4898ea7aeac None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:19.734343 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdee8f64-a6ed-4523-a5fe-51a54d108724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:19.734993 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdee8f64-a6ed-4523-a5fe-51a54d108724 None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:19.735278 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdee8f64-a6ed-4523-a5fe-51a54d108724 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:19.735591 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdee8f64-a6ed-4523-a5fe-51a54d108724 None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:19.736285 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdee8f64-a6ed-4523-a5fe-51a54d108724 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:19.737857 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdee8f64-a6ed-4523-a5fe-51a54d108724 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:19.851167 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 594/1205] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Aug 4 21:06:19 2022] POST /identity/v3/users => generated 251 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:06:21.906602 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06840c1-578c-4139-8dd1-3ee8c18dce1b None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:21.907168 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06840c1-578c-4139-8dd1-3ee8c18dce1b None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:21.907168 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06840c1-578c-4139-8dd1-3ee8c18dce1b None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:21.908569 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 612/1206] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:06:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:06:21.915829 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2c633e-9b03-4c57-96f6-cae8cd6962ff None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:21.916489 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2c633e-9b03-4c57-96f6-cae8cd6962ff None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:21.916887 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2c633e-9b03-4c57-96f6-cae8cd6962ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:21.936941 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-0d2c633e-9b03-4c57-96f6-cae8cd6962ff None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:06:21.938001 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-0d2c633e-9b03-4c57-96f6-cae8cd6962ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:22.008660 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 595/1207] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:06:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:06:22.032354 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06840c1-578c-4139-8dd1-3ee8c18dce1b None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:22.139153 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06840c1-578c-4139-8dd1-3ee8c18dce1b None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:22.180902 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eda9896-1249-4d27-b2ff-1a6fc363ca39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:22.180902 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eda9896-1249-4d27-b2ff-1a6fc363ca39 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:22.181331 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eda9896-1249-4d27-b2ff-1a6fc363ca39 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:22.181554 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eda9896-1249-4d27-b2ff-1a6fc363ca39 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:22.191909 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eda9896-1249-4d27-b2ff-1a6fc363ca39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:06:22.196814 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-3eda9896-1249-4d27-b2ff-1a6fc363ca39 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:06:22.196814 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:06:22.196814 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:06:22.196814 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:06:22.196814 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:06:22.196814 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:06:22.196814 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:06:22.196814 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:06:22.196814 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:06:22.196814 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:06:22.196814 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:06:22.196814 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:06:22.196814 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:06:22.196814 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:06:22.196814 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:06:22.196814 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:06:22.197923 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:06:22.197923 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:06:22.197923 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:06:22.197923 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:06:22.197923 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:06:22.197923 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:06:22.197923 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:06:22.197923 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:06:22.197923 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:06:22.197923 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:06:22.197923 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:06:22.197923 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:06:22.197923 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:06:22.197923 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:06:22.197923 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:06:22.197923 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:06:22.198963 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:06:22.198963 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:06:22.198963 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:06:22.198963 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:06:22.198963 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:06:22.198963 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:06:22.198963 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:06:22.198963 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:06:22.198963 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:06:22.198963 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:06:22.198963 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:06:22.198963 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:06:22.198963 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 613/1208] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Aug 4 21:06:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:06:22.204195 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2c633e-9b03-4c57-96f6-cae8cd6962ff None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:22.245115 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:22.302148 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2c633e-9b03-4c57-96f6-cae8cd6962ff None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:22.338919 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287f49d6-99bc-44a9-94a3-7ed48d283314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:22.339637 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287f49d6-99bc-44a9-94a3-7ed48d283314 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:22.339921 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287f49d6-99bc-44a9-94a3-7ed48d283314 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:22.340192 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287f49d6-99bc-44a9-94a3-7ed48d283314 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:22.340907 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287f49d6-99bc-44a9-94a3-7ed48d283314 None admin] RBAC: Adding query filter params (name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:06:22.341209 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287f49d6-99bc-44a9-94a3-7ed48d283314 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:22.342582 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287f49d6-99bc-44a9-94a3-7ed48d283314 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:22.351850 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 596/1209] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Aug 4 21:06:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:06:22.357252 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eda9896-1249-4d27-b2ff-1a6fc363ca39 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:22.408136 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:22.460751 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eda9896-1249-4d27-b2ff-1a6fc363ca39 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:22.495169 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7f88be-1dce-4388-a166-a43f3fb47c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:22.496135 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7f88be-1dce-4388-a166-a43f3fb47c99 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:22.496540 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7f88be-1dce-4388-a166-a43f3fb47c99 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:22.496944 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7f88be-1dce-4388-a166-a43f3fb47c99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:22.512990 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7f88be-1dce-4388-a166-a43f3fb47c99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:22.515199 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7f88be-1dce-4388-a166-a43f3fb47c99 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:22.534923 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-2d7f88be-1dce-4388-a166-a43f3fb47c99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:06:22.534923 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:06:22.534923 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:06:22.534923 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:06:22.534923 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:06:22.534923 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:06:22.534923 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:06:22.534923 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:06:22.534923 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:06:22.534923 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:06:22.534923 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:06:22.534923 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:06:22.534923 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:06:22.534923 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:06:22.534923 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:06:22.536071 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:06:22.536071 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:06:22.536071 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:06:22.536071 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:06:22.536071 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:06:22.536071 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:06:22.536071 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:06:22.536071 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:06:22.536071 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:06:22.536071 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:06:22.536071 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:06:22.536071 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:06:22.536071 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:06:22.536071 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:06:22.536071 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:06:22.536071 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:06:22.536071 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:06:22.537100 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:06:22.537100 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:06:22.537100 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:06:22.537100 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:06:22.537100 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 614/1210] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:06:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:06:22.542350 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287f49d6-99bc-44a9-94a3-7ed48d283314 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:22.649983 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:22.653064 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287f49d6-99bc-44a9-94a3-7ed48d283314 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:22.690680 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe9fa3a-05ac-403f-b069-2a76c4e08fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:22.691307 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe9fa3a-05ac-403f-b069-2a76c4e08fda None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:22.691615 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe9fa3a-05ac-403f-b069-2a76c4e08fda None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:22.691886 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe9fa3a-05ac-403f-b069-2a76c4e08fda None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:22.692687 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe9fa3a-05ac-403f-b069-2a76c4e08fda None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:06:22.693026 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe9fa3a-05ac-403f-b069-2a76c4e08fda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:22.694414 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe9fa3a-05ac-403f-b069-2a76c4e08fda None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:22.706602 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 597/1211] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:06:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:06:22.712466 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7f88be-1dce-4388-a166-a43f3fb47c99 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:22.812023 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7f88be-1dce-4388-a166-a43f3fb47c99 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:22.813556 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:22.848531 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c4b253-2313-4644-81aa-84381696d1f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:22.849171 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c4b253-2313-4644-81aa-84381696d1f1 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:22.849460 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c4b253-2313-4644-81aa-84381696d1f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:22.849731 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c4b253-2313-4644-81aa-84381696d1f1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:22.856902 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c4b253-2313-4644-81aa-84381696d1f1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:22.858398 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c4b253-2313-4644-81aa-84381696d1f1 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:22.871637 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-a4c4b253-2313-4644-81aa-84381696d1f1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 04 21:06:22.871637 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:06:22.871637 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:06:22.871637 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:06:22.871637 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:06:22.871637 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:06:22.871637 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:06:22.871637 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:06:22.871637 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:06:22.871637 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:06:22.871637 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:06:22.871637 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:06:22.871637 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 21:06:22.871637 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 21:06:22.871637 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 21:06:22.871637 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 21:06:22.872798 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:06:22.872798 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:06:22.872798 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 21:06:22.872798 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:06:22.872798 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 21:06:22.872798 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:06:22.872798 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:06:22.872798 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:06:22.872798 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:06:22.872798 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:06:22.872798 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:06:22.872798 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:06:22.872798 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:06:22.872798 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:06:22.872798 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:06:22.872798 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:06:22.872798 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:06:22.872798 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:06:22.874168 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:06:22.874168 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:06:22.874168 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 21:06:22.874168 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 21:06:22.874168 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 21:06:22.874168 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 21:06:22.874168 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 21:06:22.874168 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 21:06:22.874168 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 04 21:06:22.874168 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:06:22.874168 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 615/1212] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:06:22 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:06:22.879401 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe9fa3a-05ac-403f-b069-2a76c4e08fda None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:22.984736 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe9fa3a-05ac-403f-b069-2a76c4e08fda None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:23.020726 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd893006-eaf7-43fa-a6ac-7b831dd54e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:23.021360 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd893006-eaf7-43fa-a6ac-7b831dd54e60 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:23.021646 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd893006-eaf7-43fa-a6ac-7b831dd54e60 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:23.021947 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd893006-eaf7-43fa-a6ac-7b831dd54e60 None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:23.022723 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd893006-eaf7-43fa-a6ac-7b831dd54e60 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:06:23.023031 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd893006-eaf7-43fa-a6ac-7b831dd54e60 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:23.024560 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd893006-eaf7-43fa-a6ac-7b831dd54e60 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:23.042035 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 598/1213] 10.222.0.25 () {56 vars in 1143 bytes} [Thu Aug 4 21:06:22 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:06:23.047915 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c4b253-2313-4644-81aa-84381696d1f1 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:23.151481 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c4b253-2313-4644-81aa-84381696d1f1 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:23.185764 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dc7a30-3953-449b-baa6-0f7fb8d20886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:23.186387 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dc7a30-3953-449b-baa6-0f7fb8d20886 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:23.186667 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dc7a30-3953-449b-baa6-0f7fb8d20886 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:23.186941 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dc7a30-3953-449b-baa6-0f7fb8d20886 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:23.217126 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:23.255765 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dc7a30-3953-449b-baa6-0f7fb8d20886 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:23.257228 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dc7a30-3953-449b-baa6-0f7fb8d20886 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:23.353121 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 616/1214] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Aug 4 21:06:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 21:06:23.359033 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd893006-eaf7-43fa-a6ac-7b831dd54e60 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:23.378336 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:23.468141 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd893006-eaf7-43fa-a6ac-7b831dd54e60 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:23.505537 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54f6c84-e902-49e9-b9a5-e5c94cf893f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:23.506237 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54f6c84-e902-49e9-b9a5-e5c94cf893f6 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:23.506466 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54f6c84-e902-49e9-b9a5-e5c94cf893f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:23.506745 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54f6c84-e902-49e9-b9a5-e5c94cf893f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:23.516473 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54f6c84-e902-49e9-b9a5-e5c94cf893f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:23.517927 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54f6c84-e902-49e9-b9a5-e5c94cf893f6 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:23.532235 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-d54f6c84-e902-49e9-b9a5-e5c94cf893f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:06:23.532235 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:06:23.532235 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:06:23.532235 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:06:23.532235 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:06:23.532235 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:06:23.532235 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:06:23.532235 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:06:23.532235 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:06:23.532235 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:06:23.532235 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:06:23.532235 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:06:23.532235 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:06:23.532235 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:06:23.532235 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:06:23.535957 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:06:23.535957 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:06:23.535957 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:06:23.535957 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:06:23.535957 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:06:23.535957 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:06:23.535957 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:06:23.535957 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:06:23.535957 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:06:23.535957 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:06:23.535957 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:06:23.535957 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:06:23.535957 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:06:23.535957 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:06:23.535957 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:06:23.535957 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:06:23.535957 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:06:23.536969 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:06:23.536969 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:06:23.536969 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:06:23.536969 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:06:23.536969 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 599/1215] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:06:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:06:23.539912 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dc7a30-3953-449b-baa6-0f7fb8d20886 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:23.606260 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:23.654736 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dc7a30-3953-449b-baa6-0f7fb8d20886 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:23.689740 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1c9f34-386d-4a1b-8907-1e694b22147e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:23.690454 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1c9f34-386d-4a1b-8907-1e694b22147e None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:23.690731 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1c9f34-386d-4a1b-8907-1e694b22147e None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:23.691000 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1c9f34-386d-4a1b-8907-1e694b22147e None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:23.691738 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1c9f34-386d-4a1b-8907-1e694b22147e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:06:23.692040 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1c9f34-386d-4a1b-8907-1e694b22147e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:23.693391 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1c9f34-386d-4a1b-8907-1e694b22147e None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:23.705192 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 617/1216] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:06:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:06:23.711576 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54f6c84-e902-49e9-b9a5-e5c94cf893f6 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:23.781225 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:23.813491 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54f6c84-e902-49e9-b9a5-e5c94cf893f6 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:23.856623 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc62b5a0-5a3e-4558-a94f-0e568a1ef154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:23.857266 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc62b5a0-5a3e-4558-a94f-0e568a1ef154 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:23.857546 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc62b5a0-5a3e-4558-a94f-0e568a1ef154 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:23.858219 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc62b5a0-5a3e-4558-a94f-0e568a1ef154 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:23.867085 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc62b5a0-5a3e-4558-a94f-0e568a1ef154 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:23.868627 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc62b5a0-5a3e-4558-a94f-0e568a1ef154 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:23.881148 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-dc62b5a0-5a3e-4558-a94f-0e568a1ef154 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:06:23.881148 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:06:23.881148 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:06:23.881148 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:06:23.881148 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:06:23.881148 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:06:23.881148 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:06:23.881148 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:06:23.881148 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:06:23.881148 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:06:23.881148 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:06:23.881148 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:06:23.881148 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 21:06:23.881148 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 21:06:23.881148 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 21:06:23.882323 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 21:06:23.882323 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:06:23.882323 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:06:23.882323 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:06:23.882323 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:06:23.882323 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:06:23.882323 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:06:23.882323 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:06:23.882323 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:06:23.882323 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:06:23.882323 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:06:23.882323 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:06:23.882323 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:06:23.882323 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:06:23.882323 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:06:23.882323 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:06:23.882323 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:06:23.883291 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 21:06:23.883291 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 21:06:23.883291 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 21:06:23.883291 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 21:06:23.883291 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 21:06:23.883291 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 21:06:23.883291 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:06:23.883291 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:06:23.883291 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 600/1217] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Aug 4 21:06:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:06:23.888286 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1c9f34-386d-4a1b-8907-1e694b22147e None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:23.998719 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1c9f34-386d-4a1b-8907-1e694b22147e None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:24.025872 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:24.037236 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3204ff13-9aba-4cd9-9e80-f6e814ecce3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:24.038136 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3204ff13-9aba-4cd9-9e80-f6e814ecce3f None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:24.038431 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3204ff13-9aba-4cd9-9e80-f6e814ecce3f None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:24.038697 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3204ff13-9aba-4cd9-9e80-f6e814ecce3f None admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:24.039484 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3204ff13-9aba-4cd9-9e80-f6e814ecce3f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:06:24.039794 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3204ff13-9aba-4cd9-9e80-f6e814ecce3f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:24.041232 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3204ff13-9aba-4cd9-9e80-f6e814ecce3f None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:24.052954 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 618/1218] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Aug 4 21:06:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:06:24.059461 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc62b5a0-5a3e-4558-a94f-0e568a1ef154 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:24.163136 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc62b5a0-5a3e-4558-a94f-0e568a1ef154 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:24.192948 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:24.198964 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc987a47-c135-4280-8917-a04effeaf5a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:24.202002 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc987a47-c135-4280-8917-a04effeaf5a9 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:24.202002 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc987a47-c135-4280-8917-a04effeaf5a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:24.202002 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc987a47-c135-4280-8917-a04effeaf5a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:24.202002 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc987a47-c135-4280-8917-a04effeaf5a9 None admin] RBAC: Adding query filter params (role.id=4eb30338d61e47808b51e050836643ea, scope.project.id=17dc3d39e2c048ca9070588f5717a578, user.id=67d44a0098ab4c36b14b36e9e59ae75c) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:06:24.202002 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc987a47-c135-4280-8917-a04effeaf5a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:24.202817 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc987a47-c135-4280-8917-a04effeaf5a9 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:24.209182 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 601/1219] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Aug 4 21:06:24 2022] GET /identity/v3/role_assignments?user.id=67d44a0098ab4c36b14b36e9e59ae75c&scope.project.id=17dc3d39e2c048ca9070588f5717a578&role.id=4eb30338d61e47808b51e050836643ea => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:06:26.208054 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f435c42-1f58-47d5-843e-f747e6100cd9 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:26.208567 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f435c42-1f58-47d5-843e-f747e6100cd9 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:26.208684 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f435c42-1f58-47d5-843e-f747e6100cd9 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:26.209955 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 619/1220] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:06:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:06:26.215100 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a8e62a-4251-45df-b152-18d8b3618659 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:26.215561 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a8e62a-4251-45df-b152-18d8b3618659 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:26.215695 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a8e62a-4251-45df-b152-18d8b3618659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:26.235226 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-94a8e62a-4251-45df-b152-18d8b3618659 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:06:26.236453 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-94a8e62a-4251-45df-b152-18d8b3618659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:26.307388 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 602/1221] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:06:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:06:26.329999 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f435c42-1f58-47d5-843e-f747e6100cd9 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:26.432683 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f435c42-1f58-47d5-843e-f747e6100cd9 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:26.466599 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de04f189-b70d-4bf4-b2ba-8927c69874d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:26.467169 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de04f189-b70d-4bf4-b2ba-8927c69874d8 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:26.467483 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de04f189-b70d-4bf4-b2ba-8927c69874d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:26.467758 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de04f189-b70d-4bf4-b2ba-8927c69874d8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:26.477765 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de04f189-b70d-4bf4-b2ba-8927c69874d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:06:26.482800 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-de04f189-b70d-4bf4-b2ba-8927c69874d8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:06:26.482800 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:06:26.482800 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:06:26.482800 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:06:26.482800 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:06:26.482800 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:06:26.482800 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:06:26.482800 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:06:26.482800 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:06:26.482800 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:06:26.482800 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:06:26.482800 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:06:26.482800 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:06:26.482800 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:06:26.482800 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:06:26.482800 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:06:26.483948 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:06:26.483948 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:06:26.483948 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:06:26.483948 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:06:26.483948 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:06:26.483948 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:06:26.483948 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:06:26.483948 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:06:26.483948 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:06:26.483948 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:06:26.483948 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:06:26.483948 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:06:26.483948 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:06:26.483948 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:06:26.483948 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:06:26.483948 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:06:26.485172 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:06:26.485172 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:06:26.485172 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:06:26.485172 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:06:26.485172 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:06:26.485172 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:06:26.485172 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:06:26.485172 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:06:26.485172 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:06:26.485172 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:06:26.485172 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:06:26.485172 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:06:26.485172 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 620/1222] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Aug 4 21:06:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:06:26.490193 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a8e62a-4251-45df-b152-18d8b3618659 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:26.588215 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a8e62a-4251-45df-b152-18d8b3618659 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:26.613720 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:26.631878 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b940b5a1-e11c-4c07-8fd2-4b9084e872e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:26.632110 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b940b5a1-e11c-4c07-8fd2-4b9084e872e5 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:26.632402 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b940b5a1-e11c-4c07-8fd2-4b9084e872e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:26.632672 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b940b5a1-e11c-4c07-8fd2-4b9084e872e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:26.633385 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b940b5a1-e11c-4c07-8fd2-4b9084e872e5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:06:26.633689 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b940b5a1-e11c-4c07-8fd2-4b9084e872e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:26.635121 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b940b5a1-e11c-4c07-8fd2-4b9084e872e5 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:26.645591 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 603/1223] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Aug 4 21:06:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:06:26.651573 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de04f189-b70d-4bf4-b2ba-8927c69874d8 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:26.696401 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:26.751240 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de04f189-b70d-4bf4-b2ba-8927c69874d8 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:26.784030 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2204ba5a-a80a-41e6-a061-f7e1a6e803f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:26.784659 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2204ba5a-a80a-41e6-a061-f7e1a6e803f6 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:26.784936 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2204ba5a-a80a-41e6-a061-f7e1a6e803f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:26.785214 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2204ba5a-a80a-41e6-a061-f7e1a6e803f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:26.795828 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2204ba5a-a80a-41e6-a061-f7e1a6e803f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:26.797468 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2204ba5a-a80a-41e6-a061-f7e1a6e803f6 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:26.811987 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-2204ba5a-a80a-41e6-a061-f7e1a6e803f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:06:26.811987 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:06:26.811987 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:06:26.811987 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:06:26.811987 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:06:26.811987 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:06:26.811987 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:06:26.811987 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:06:26.811987 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:06:26.811987 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:06:26.811987 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:06:26.811987 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:06:26.811987 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:06:26.811987 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:06:26.811987 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:06:26.813067 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:06:26.813067 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:06:26.813067 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:06:26.813067 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:06:26.813067 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:06:26.813067 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:06:26.813067 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:06:26.813067 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:06:26.813067 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:06:26.813067 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:06:26.813067 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:06:26.813067 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:06:26.813067 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:06:26.813067 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:06:26.813067 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:06:26.813067 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:06:26.813067 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:06:26.814161 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:06:26.814161 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:06:26.814161 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:06:26.814161 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:06:26.814161 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 621/1224] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:06:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:06:26.819675 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b940b5a1-e11c-4c07-8fd2-4b9084e872e5 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:26.921482 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b940b5a1-e11c-4c07-8fd2-4b9084e872e5 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:26.956401 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170c4775-51f2-4de6-950c-5e8cf4035028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:26.957037 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170c4775-51f2-4de6-950c-5e8cf4035028 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:26.957321 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170c4775-51f2-4de6-950c-5e8cf4035028 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:26.957595 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170c4775-51f2-4de6-950c-5e8cf4035028 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:26.958284 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170c4775-51f2-4de6-950c-5e8cf4035028 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:06:26.958586 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170c4775-51f2-4de6-950c-5e8cf4035028 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:26.960003 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170c4775-51f2-4de6-950c-5e8cf4035028 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:26.972974 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 604/1225] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:06:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:06:26.978895 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2204ba5a-a80a-41e6-a061-f7e1a6e803f6 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:27.075573 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:27.082735 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2204ba5a-a80a-41e6-a061-f7e1a6e803f6 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:27.119758 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777cf97a-b722-4bf6-afe7-a944f351be9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:27.120402 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777cf97a-b722-4bf6-afe7-a944f351be9d None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:27.120686 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777cf97a-b722-4bf6-afe7-a944f351be9d None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:27.120957 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777cf97a-b722-4bf6-afe7-a944f351be9d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:27.128948 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777cf97a-b722-4bf6-afe7-a944f351be9d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:27.131500 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777cf97a-b722-4bf6-afe7-a944f351be9d None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:27.143824 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-777cf97a-b722-4bf6-afe7-a944f351be9d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 04 21:06:27.143824 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:06:27.143824 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:06:27.143824 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:06:27.143824 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:06:27.143824 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:06:27.143824 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:06:27.143824 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:06:27.143824 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:06:27.143824 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:06:27.143824 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:06:27.143824 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:06:27.143824 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 21:06:27.143824 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 21:06:27.143824 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 21:06:27.143824 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 21:06:27.148583 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:06:27.148583 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:06:27.148583 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 21:06:27.148583 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:06:27.148583 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 21:06:27.148583 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:06:27.148583 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:06:27.148583 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:06:27.148583 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:06:27.148583 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:06:27.148583 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:06:27.148583 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:06:27.148583 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:06:27.148583 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:06:27.148583 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:06:27.148583 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:06:27.148583 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:06:27.148583 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:06:27.150401 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:06:27.150401 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:06:27.150401 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 21:06:27.150401 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 21:06:27.150401 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 21:06:27.150401 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 21:06:27.150401 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 21:06:27.150401 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 21:06:27.150401 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 04 21:06:27.150401 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:06:27.150401 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 622/1226] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:06:26 2022] GET /identity/v3/users/placement => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:06:27.152487 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170c4775-51f2-4de6-950c-5e8cf4035028 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:27.202161 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:27.256723 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170c4775-51f2-4de6-950c-5e8cf4035028 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:27.291528 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3ba19d-8bc9-47d3-a1b0-d247072f1087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:27.292696 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3ba19d-8bc9-47d3-a1b0-d247072f1087 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:27.292987 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3ba19d-8bc9-47d3-a1b0-d247072f1087 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:27.293297 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3ba19d-8bc9-47d3-a1b0-d247072f1087 None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:27.294102 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3ba19d-8bc9-47d3-a1b0-d247072f1087 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:06:27.294491 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3ba19d-8bc9-47d3-a1b0-d247072f1087 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:27.295910 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3ba19d-8bc9-47d3-a1b0-d247072f1087 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:27.313908 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 605/1227] 10.222.0.25 () {56 vars in 1143 bytes} [Thu Aug 4 21:06:27 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:06:27.320254 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777cf97a-b722-4bf6-afe7-a944f351be9d None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:27.420527 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777cf97a-b722-4bf6-afe7-a944f351be9d None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:27.457941 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c641554f-a076-484d-ab4b-2043d2052963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:27.459031 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c641554f-a076-484d-ab4b-2043d2052963 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:27.459575 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c641554f-a076-484d-ab4b-2043d2052963 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:27.460063 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c641554f-a076-484d-ab4b-2043d2052963 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:27.475883 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c641554f-a076-484d-ab4b-2043d2052963 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:27.477326 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c641554f-a076-484d-ab4b-2043d2052963 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:27.491895 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-c641554f-a076-484d-ab4b-2043d2052963 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:06:27.491895 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:06:27.491895 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:06:27.491895 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:06:27.491895 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:06:27.491895 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:06:27.491895 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:06:27.491895 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:06:27.491895 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:06:27.491895 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:06:27.491895 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:06:27.491895 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:06:27.491895 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:06:27.491895 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:06:27.491895 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:06:27.492880 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:06:27.492880 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:06:27.492880 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:06:27.492880 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:06:27.492880 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:06:27.492880 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:06:27.492880 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:06:27.492880 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:06:27.492880 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:06:27.492880 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:06:27.492880 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:06:27.492880 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:06:27.492880 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:06:27.492880 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:06:27.492880 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:06:27.492880 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:06:27.492880 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:06:27.493932 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:06:27.493932 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:06:27.493932 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:06:27.493932 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:06:27.493932 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 623/1228] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:06:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:06:27.501892 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3ba19d-8bc9-47d3-a1b0-d247072f1087 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:27.614678 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3ba19d-8bc9-47d3-a1b0-d247072f1087 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:27.617134 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:27.650213 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f713a3e-75d7-4f7f-aeae-71361696f9c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:27.650833 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f713a3e-75d7-4f7f-aeae-71361696f9c3 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:27.651115 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f713a3e-75d7-4f7f-aeae-71361696f9c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:27.651417 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f713a3e-75d7-4f7f-aeae-71361696f9c3 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:27.652108 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f713a3e-75d7-4f7f-aeae-71361696f9c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:06:27.652411 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f713a3e-75d7-4f7f-aeae-71361696f9c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:27.653838 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f713a3e-75d7-4f7f-aeae-71361696f9c3 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:27.665582 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 606/1229] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:06:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:06:27.672159 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c641554f-a076-484d-ab4b-2043d2052963 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:27.682292 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:27.778450 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c641554f-a076-484d-ab4b-2043d2052963 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:27.812718 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72a802c-9821-4d9d-a644-4ebd7e51802e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:27.813359 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a802c-9821-4d9d-a644-4ebd7e51802e None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:27.813640 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a802c-9821-4d9d-a644-4ebd7e51802e None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:27.813915 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a802c-9821-4d9d-a644-4ebd7e51802e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:27.820482 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72a802c-9821-4d9d-a644-4ebd7e51802e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:27.821916 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72a802c-9821-4d9d-a644-4ebd7e51802e None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:27.833173 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-d72a802c-9821-4d9d-a644-4ebd7e51802e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:06:27.833173 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:06:27.833173 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:06:27.833173 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:06:27.833173 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:06:27.833173 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:06:27.833173 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:06:27.833173 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:06:27.833173 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:06:27.833173 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:06:27.833173 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:06:27.833173 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:06:27.833173 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 21:06:27.833173 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 21:06:27.833173 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 21:06:27.834249 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 21:06:27.834249 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:06:27.834249 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:06:27.834249 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:06:27.834249 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:06:27.834249 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:06:27.834249 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:06:27.834249 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:06:27.834249 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:06:27.834249 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:06:27.834249 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:06:27.834249 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:06:27.834249 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:06:27.834249 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:06:27.834249 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:06:27.834249 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:06:27.834249 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:06:27.835378 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 21:06:27.835378 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 21:06:27.835378 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 21:06:27.835378 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 21:06:27.835378 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 21:06:27.835378 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 21:06:27.835378 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:06:27.835378 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:06:27.835378 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 624/1230] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Aug 4 21:06:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:06:27.840342 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f713a3e-75d7-4f7f-aeae-71361696f9c3 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:27.944842 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f713a3e-75d7-4f7f-aeae-71361696f9c3 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:27.977886 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d8d049-d11e-46d7-ad72-7c4ac3d87ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:27.978492 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d8d049-d11e-46d7-ad72-7c4ac3d87ed8 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:27.978776 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d8d049-d11e-46d7-ad72-7c4ac3d87ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:27.979048 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d8d049-d11e-46d7-ad72-7c4ac3d87ed8 None admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:27.979937 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d8d049-d11e-46d7-ad72-7c4ac3d87ed8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:06:27.980246 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d8d049-d11e-46d7-ad72-7c4ac3d87ed8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:27.981681 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d8d049-d11e-46d7-ad72-7c4ac3d87ed8 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:27.994012 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 607/1231] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Aug 4 21:06:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:06:28.000120 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72a802c-9821-4d9d-a644-4ebd7e51802e None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:28.064632 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:28.098750 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72a802c-9821-4d9d-a644-4ebd7e51802e None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:28.132160 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec23c15-0d28-4cf4-a5fa-f8f5f853cdcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:28.132832 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec23c15-0d28-4cf4-a5fa-f8f5f853cdcf None admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:28.133119 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec23c15-0d28-4cf4-a5fa-f8f5f853cdcf None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:28.133399 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec23c15-0d28-4cf4-a5fa-f8f5f853cdcf None admin] PATH_INFO: `/v3/projects/17dc3d39e2c048ca9070588f5717a578/users/67d44a0098ab4c36b14b36e9e59ae75c/roles/4eb30338d61e47808b51e050836643ea` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:28.148282 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec23c15-0d28-4cf4-a5fa-f8f5f853cdcf None admin] RBAC: Authorizing `identity:create_grant(project_id=17dc3d39e2c048ca9070588f5717a578, user_id=67d44a0098ab4c36b14b36e9e59ae75c, role_id=4eb30338d61e47808b51e050836643ea)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:28.149716 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec23c15-0d28-4cf4-a5fa-f8f5f853cdcf None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:28.274635 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 625/1232] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Aug 4 21:06:27 2022] PUT /identity/v3/projects/17dc3d39e2c048ca9070588f5717a578/users/67d44a0098ab4c36b14b36e9e59ae75c/roles/4eb30338d61e47808b51e050836643ea => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:06:30.249687 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd068cea-c110-43ce-9288-fef4d7fa40e2 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:30.250172 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd068cea-c110-43ce-9288-fef4d7fa40e2 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:30.250286 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd068cea-c110-43ce-9288-fef4d7fa40e2 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:30.251670 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 608/1233] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:06:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:06:30.257328 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b169b5-b172-43fe-ac63-3e5a96bc22a4 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:30.257740 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b169b5-b172-43fe-ac63-3e5a96bc22a4 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:30.257880 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b169b5-b172-43fe-ac63-3e5a96bc22a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:30.276353 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-33b169b5-b172-43fe-ac63-3e5a96bc22a4 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:06:30.277538 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-33b169b5-b172-43fe-ac63-3e5a96bc22a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:30.344272 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 626/1234] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:06:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:06:30.367420 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd068cea-c110-43ce-9288-fef4d7fa40e2 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:30.395972 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:30.473905 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd068cea-c110-43ce-9288-fef4d7fa40e2 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:30.507202 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74dc923-1d39-4a02-9d5a-49a0a484b91f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:30.507805 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74dc923-1d39-4a02-9d5a-49a0a484b91f None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:30.508082 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74dc923-1d39-4a02-9d5a-49a0a484b91f None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:30.508348 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74dc923-1d39-4a02-9d5a-49a0a484b91f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:30.517952 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74dc923-1d39-4a02-9d5a-49a0a484b91f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:06:30.523158 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-c74dc923-1d39-4a02-9d5a-49a0a484b91f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:06:30.523158 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:06:30.523158 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:06:30.523158 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:06:30.523158 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:06:30.523158 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:06:30.523158 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:06:30.523158 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:06:30.523158 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:06:30.523158 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:06:30.523158 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:06:30.523158 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:06:30.523158 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:06:30.523158 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:06:30.523158 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:06:30.523158 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:06:30.524364 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:06:30.524364 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:06:30.524364 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:06:30.524364 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:06:30.524364 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:06:30.524364 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:06:30.524364 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:06:30.524364 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:06:30.524364 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:06:30.524364 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:06:30.524364 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:06:30.524364 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:06:30.524364 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:06:30.524364 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:06:30.524364 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:06:30.524364 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:06:30.525235 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:06:30.525235 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:06:30.525235 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:06:30.525235 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:06:30.525235 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:06:30.525235 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:06:30.525235 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:06:30.525235 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:06:30.525235 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:06:30.525235 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:06:30.525235 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:06:30.525235 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:06:30.525902 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 609/1235] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Aug 4 21:06:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:06:30.530451 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b169b5-b172-43fe-ac63-3e5a96bc22a4 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:30.576873 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:30.629952 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b169b5-b172-43fe-ac63-3e5a96bc22a4 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:30.662393 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe0d9ad-bc92-4f77-a1f5-78ad3b274d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:30.663265 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe0d9ad-bc92-4f77-a1f5-78ad3b274d89 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:30.663577 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe0d9ad-bc92-4f77-a1f5-78ad3b274d89 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:30.663849 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe0d9ad-bc92-4f77-a1f5-78ad3b274d89 None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:30.664560 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe0d9ad-bc92-4f77-a1f5-78ad3b274d89 None admin] RBAC: Adding query filter params (name=service) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:06:30.664885 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe0d9ad-bc92-4f77-a1f5-78ad3b274d89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:30.666357 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe0d9ad-bc92-4f77-a1f5-78ad3b274d89 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:30.674219 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 627/1236] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Aug 4 21:06:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:06:30.680451 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74dc923-1d39-4a02-9d5a-49a0a484b91f None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:30.765362 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:30.780957 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74dc923-1d39-4a02-9d5a-49a0a484b91f None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:30.814252 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f9b685-4a2a-4b5c-a992-be6b93c6374c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:30.814889 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f9b685-4a2a-4b5c-a992-be6b93c6374c None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:30.815200 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f9b685-4a2a-4b5c-a992-be6b93c6374c None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:30.815507 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f9b685-4a2a-4b5c-a992-be6b93c6374c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:30.825346 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f9b685-4a2a-4b5c-a992-be6b93c6374c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:30.826873 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f9b685-4a2a-4b5c-a992-be6b93c6374c None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:30.841079 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-90f9b685-4a2a-4b5c-a992-be6b93c6374c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:06:30.841079 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:06:30.841079 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:06:30.841079 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:06:30.841079 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:06:30.841079 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:06:30.841079 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:06:30.841079 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:06:30.841079 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:06:30.841079 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:06:30.841079 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:06:30.841079 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:06:30.841079 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:06:30.841079 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:06:30.841079 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:06:30.842139 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:06:30.842139 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:06:30.842139 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:06:30.842139 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:06:30.842139 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:06:30.842139 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:06:30.842139 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:06:30.842139 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:06:30.842139 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:06:30.842139 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:06:30.842139 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:06:30.842139 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:06:30.842139 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:06:30.842139 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:06:30.842139 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:06:30.842139 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:06:30.842139 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:06:30.843459 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:06:30.843459 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:06:30.843459 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:06:30.843459 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:06:30.843459 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 610/1237] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:06:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:06:30.848481 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe0d9ad-bc92-4f77-a1f5-78ad3b274d89 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:30.950119 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe0d9ad-bc92-4f77-a1f5-78ad3b274d89 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:30.959693 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:30.983998 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7462947e-730c-4512-8c59-6c7b1981f44b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:30.984610 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7462947e-730c-4512-8c59-6c7b1981f44b None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:30.984890 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7462947e-730c-4512-8c59-6c7b1981f44b None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:30.985161 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7462947e-730c-4512-8c59-6c7b1981f44b None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:30.985839 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7462947e-730c-4512-8c59-6c7b1981f44b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:06:30.986138 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7462947e-730c-4512-8c59-6c7b1981f44b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:30.988121 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7462947e-730c-4512-8c59-6c7b1981f44b None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:30.999377 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 628/1238] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:06:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:06:31.005376 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f9b685-4a2a-4b5c-a992-be6b93c6374c None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:31.110245 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f9b685-4a2a-4b5c-a992-be6b93c6374c None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:31.143132 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb989be-0ef1-4e1d-babe-ee3045920db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:31.144222 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb989be-0ef1-4e1d-babe-ee3045920db7 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:31.144499 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb989be-0ef1-4e1d-babe-ee3045920db7 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:31.144767 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb989be-0ef1-4e1d-babe-ee3045920db7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:31.147356 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:31.152311 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb989be-0ef1-4e1d-babe-ee3045920db7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:31.153763 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb989be-0ef1-4e1d-babe-ee3045920db7 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:31.165970 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-5cb989be-0ef1-4e1d-babe-ee3045920db7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 04 21:06:31.165970 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:06:31.165970 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:06:31.165970 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:06:31.165970 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:06:31.165970 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:06:31.165970 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:06:31.165970 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:06:31.165970 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:06:31.165970 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:06:31.165970 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:06:31.165970 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:06:31.165970 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 21:06:31.165970 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 21:06:31.165970 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 21:06:31.165970 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 21:06:31.167122 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:06:31.167122 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:06:31.167122 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 21:06:31.167122 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:06:31.167122 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 21:06:31.167122 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:06:31.167122 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:06:31.167122 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:06:31.167122 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:06:31.167122 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:06:31.167122 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:06:31.167122 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:06:31.167122 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:06:31.167122 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:06:31.167122 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:06:31.167122 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:06:31.167122 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:06:31.167122 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:06:31.168565 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:06:31.168565 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:06:31.168565 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 21:06:31.168565 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 21:06:31.168565 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 21:06:31.168565 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 21:06:31.168565 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 21:06:31.168565 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 21:06:31.168565 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 04 21:06:31.168565 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:06:31.168565 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 611/1239] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:06:31 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:06:31.172892 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7462947e-730c-4512-8c59-6c7b1981f44b None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:31.270663 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7462947e-730c-4512-8c59-6c7b1981f44b None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:31.304081 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154feab2-3f68-4097-94fc-e1bac65566f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:31.304692 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154feab2-3f68-4097-94fc-e1bac65566f3 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:31.304975 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154feab2-3f68-4097-94fc-e1bac65566f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:31.305253 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154feab2-3f68-4097-94fc-e1bac65566f3 None admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:31.306031 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154feab2-3f68-4097-94fc-e1bac65566f3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:06:31.306335 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154feab2-3f68-4097-94fc-e1bac65566f3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:31.307793 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154feab2-3f68-4097-94fc-e1bac65566f3 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:31.324360 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 629/1240] 10.222.0.25 () {56 vars in 1143 bytes} [Thu Aug 4 21:06:31 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:06:31.330216 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb989be-0ef1-4e1d-babe-ee3045920db7 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:31.429256 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb989be-0ef1-4e1d-babe-ee3045920db7 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:31.462835 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1ff1bd-02bf-4a50-a000-11615d6c8c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:31.463489 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1ff1bd-02bf-4a50-a000-11615d6c8c65 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:31.463789 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1ff1bd-02bf-4a50-a000-11615d6c8c65 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:31.464062 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1ff1bd-02bf-4a50-a000-11615d6c8c65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:31.542078 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:31.548493 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1ff1bd-02bf-4a50-a000-11615d6c8c65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:31.549944 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1ff1bd-02bf-4a50-a000-11615d6c8c65 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:31.644327 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 612/1241] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Aug 4 21:06:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 21:06:31.650094 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154feab2-3f68-4097-94fc-e1bac65566f3 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:31.678051 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:31.749769 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154feab2-3f68-4097-94fc-e1bac65566f3 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:31.783997 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec04a103-757f-4921-8de4-febd436f5f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:31.784633 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec04a103-757f-4921-8de4-febd436f5f48 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:31.784911 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec04a103-757f-4921-8de4-febd436f5f48 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:31.785181 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec04a103-757f-4921-8de4-febd436f5f48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:31.795370 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec04a103-757f-4921-8de4-febd436f5f48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:31.796786 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec04a103-757f-4921-8de4-febd436f5f48 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:31.810997 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-ec04a103-757f-4921-8de4-febd436f5f48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:06:31.810997 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:06:31.810997 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:06:31.810997 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:06:31.810997 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:06:31.810997 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:06:31.810997 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:06:31.810997 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:06:31.810997 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:06:31.810997 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:06:31.810997 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:06:31.810997 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:06:31.810997 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:06:31.810997 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:06:31.810997 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:06:31.812101 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:06:31.812101 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:06:31.812101 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:06:31.812101 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:06:31.812101 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:06:31.812101 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:06:31.812101 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:06:31.812101 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:06:31.812101 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:06:31.812101 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:06:31.812101 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:06:31.812101 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:06:31.812101 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:06:31.812101 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:06:31.812101 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:06:31.812101 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:06:31.812101 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:06:31.813273 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:06:31.813273 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:06:31.813273 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:06:31.813273 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:06:31.813273 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 630/1242] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:06:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:06:31.818595 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1ff1bd-02bf-4a50-a000-11615d6c8c65 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:31.909026 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:31.914798 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1ff1bd-02bf-4a50-a000-11615d6c8c65 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:31.949256 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99834b53-bc98-41b7-8ccd-6a875a3f734c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:31.949868 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99834b53-bc98-41b7-8ccd-6a875a3f734c None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:31.950146 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99834b53-bc98-41b7-8ccd-6a875a3f734c None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:31.950416 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99834b53-bc98-41b7-8ccd-6a875a3f734c None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:31.951088 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99834b53-bc98-41b7-8ccd-6a875a3f734c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:06:31.951414 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99834b53-bc98-41b7-8ccd-6a875a3f734c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:31.952773 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99834b53-bc98-41b7-8ccd-6a875a3f734c None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:31.963563 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 613/1243] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:06:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:06:31.969313 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec04a103-757f-4921-8de4-febd436f5f48 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:32.046840 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:32.068663 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec04a103-757f-4921-8de4-febd436f5f48 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:32.101641 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e18d5b4-1b98-4e4f-847a-636e426f8a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:32.102273 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e18d5b4-1b98-4e4f-847a-636e426f8a9f None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:32.102552 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e18d5b4-1b98-4e4f-847a-636e426f8a9f None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:32.102821 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e18d5b4-1b98-4e4f-847a-636e426f8a9f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:32.109370 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e18d5b4-1b98-4e4f-847a-636e426f8a9f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:32.110802 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e18d5b4-1b98-4e4f-847a-636e426f8a9f None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:32.123344 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-1e18d5b4-1b98-4e4f-847a-636e426f8a9f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:06:32.123344 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:06:32.123344 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:06:32.123344 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:06:32.123344 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:06:32.123344 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:06:32.123344 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:06:32.123344 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:06:32.123344 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:06:32.123344 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:06:32.123344 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:06:32.123344 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:06:32.123344 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 21:06:32.123344 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 21:06:32.123344 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 21:06:32.124410 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 21:06:32.124410 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:06:32.124410 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:06:32.124410 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:06:32.124410 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:06:32.124410 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:06:32.124410 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:06:32.124410 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:06:32.124410 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:06:32.124410 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:06:32.124410 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:06:32.124410 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:06:32.124410 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:06:32.124410 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:06:32.124410 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:06:32.124410 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:06:32.124410 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:06:32.125525 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 21:06:32.125525 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 21:06:32.125525 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 21:06:32.125525 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 21:06:32.125525 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 21:06:32.125525 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 21:06:32.125525 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:06:32.125525 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:06:32.125525 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 631/1244] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Aug 4 21:06:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:06:32.131157 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99834b53-bc98-41b7-8ccd-6a875a3f734c None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:32.230634 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99834b53-bc98-41b7-8ccd-6a875a3f734c None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:32.263667 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92efec8e-d7a9-4c9f-a51a-4534ddc9b87e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:32.264290 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92efec8e-d7a9-4c9f-a51a-4534ddc9b87e None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:32.264574 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92efec8e-d7a9-4c9f-a51a-4534ddc9b87e None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:32.264848 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92efec8e-d7a9-4c9f-a51a-4534ddc9b87e None admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:32.265608 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92efec8e-d7a9-4c9f-a51a-4534ddc9b87e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:06:32.265910 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92efec8e-d7a9-4c9f-a51a-4534ddc9b87e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:32.267394 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92efec8e-d7a9-4c9f-a51a-4534ddc9b87e None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:32.279893 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 614/1245] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Aug 4 21:06:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:06:32.286513 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e18d5b4-1b98-4e4f-847a-636e426f8a9f None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:32.387833 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e18d5b4-1b98-4e4f-847a-636e426f8a9f None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:32.422322 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e088afa2-ed8b-4b5b-b49c-e3c59b332c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:32.422930 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e088afa2-ed8b-4b5b-b49c-e3c59b332c46 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:32.423209 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e088afa2-ed8b-4b5b-b49c-e3c59b332c46 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:32.423501 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e088afa2-ed8b-4b5b-b49c-e3c59b332c46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:32.424316 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e088afa2-ed8b-4b5b-b49c-e3c59b332c46 None admin] RBAC: Adding query filter params (role.id=4eb30338d61e47808b51e050836643ea, scope.project.id=17dc3d39e2c048ca9070588f5717a578, user.id=67d44a0098ab4c36b14b36e9e59ae75c) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:06:32.424652 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e088afa2-ed8b-4b5b-b49c-e3c59b332c46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:32.426069 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e088afa2-ed8b-4b5b-b49c-e3c59b332c46 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:32.431980 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 632/1246] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Aug 4 21:06:32 2022] GET /identity/v3/role_assignments?user.id=67d44a0098ab4c36b14b36e9e59ae75c&scope.project.id=17dc3d39e2c048ca9070588f5717a578&role.id=4eb30338d61e47808b51e050836643ea => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:06:34.514933 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a492cd-8544-4b6c-829b-cdb81297583c None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:34.515481 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a492cd-8544-4b6c-829b-cdb81297583c None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:34.515481 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a492cd-8544-4b6c-829b-cdb81297583c None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:34.516825 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 615/1247] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:06:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:06:34.522782 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6b4b69-cbc7-4871-be25-0ba1b97df054 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:34.523218 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6b4b69-cbc7-4871-be25-0ba1b97df054 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:34.523384 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6b4b69-cbc7-4871-be25-0ba1b97df054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:34.529765 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:34.541891 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-6f6b4b69-cbc7-4871-be25-0ba1b97df054 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:06:34.543202 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-6f6b4b69-cbc7-4871-be25-0ba1b97df054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:34.612235 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 633/1248] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:06:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:06:34.636061 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a492cd-8544-4b6c-829b-cdb81297583c None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:34.659965 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:34.742772 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a492cd-8544-4b6c-829b-cdb81297583c None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:34.777230 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e402e56-8f18-4593-9d11-37ab4954acfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:34.777804 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e402e56-8f18-4593-9d11-37ab4954acfe None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:34.778085 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e402e56-8f18-4593-9d11-37ab4954acfe None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:34.778435 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e402e56-8f18-4593-9d11-37ab4954acfe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:34.788703 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e402e56-8f18-4593-9d11-37ab4954acfe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:06:34.793628 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-8e402e56-8f18-4593-9d11-37ab4954acfe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:06:34.793628 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:06:34.793628 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:06:34.793628 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:06:34.793628 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:06:34.793628 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:06:34.793628 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:06:34.793628 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:06:34.793628 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:06:34.793628 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:06:34.793628 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:06:34.793628 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:06:34.793628 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:06:34.793628 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:06:34.793628 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:06:34.793628 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:06:34.798641 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:06:34.798641 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:06:34.798641 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:06:34.798641 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:06:34.798641 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:06:34.798641 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:06:34.798641 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:06:34.798641 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:06:34.798641 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:06:34.798641 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:06:34.798641 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:06:34.798641 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:06:34.798641 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:06:34.798641 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:06:34.798641 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:06:34.798641 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:06:34.799699 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:06:34.799699 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:06:34.799699 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:06:34.799699 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:06:34.799699 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:06:34.799699 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:06:34.799699 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:06:34.799699 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:06:34.799699 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:06:34.799699 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:06:34.799699 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:06:34.799699 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:06:34.799699 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 616/1249] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Aug 4 21:06:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:06:34.800870 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6b4b69-cbc7-4871-be25-0ba1b97df054 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:34.900283 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6b4b69-cbc7-4871-be25-0ba1b97df054 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:34.929217 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:34.934026 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570e6304-c4e2-4a76-ae80-36367d0a1dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:34.934644 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570e6304-c4e2-4a76-ae80-36367d0a1dd8 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:34.934933 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570e6304-c4e2-4a76-ae80-36367d0a1dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:34.935205 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570e6304-c4e2-4a76-ae80-36367d0a1dd8 None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:34.936015 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570e6304-c4e2-4a76-ae80-36367d0a1dd8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:06:34.936326 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570e6304-c4e2-4a76-ae80-36367d0a1dd8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:34.937774 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570e6304-c4e2-4a76-ae80-36367d0a1dd8 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:34.946172 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 634/1250] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Aug 4 21:06:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:06:34.951936 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e402e56-8f18-4593-9d11-37ab4954acfe None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:35.029404 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:35.051751 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e402e56-8f18-4593-9d11-37ab4954acfe None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:35.085376 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fd92a3-3e30-4780-8f23-1f2dd74dabc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:35.086014 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd92a3-3e30-4780-8f23-1f2dd74dabc4 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:35.086335 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd92a3-3e30-4780-8f23-1f2dd74dabc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:35.086611 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd92a3-3e30-4780-8f23-1f2dd74dabc4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:35.096637 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fd92a3-3e30-4780-8f23-1f2dd74dabc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:35.098148 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fd92a3-3e30-4780-8f23-1f2dd74dabc4 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:35.113447 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-87fd92a3-3e30-4780-8f23-1f2dd74dabc4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:06:35.113447 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:06:35.113447 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:06:35.113447 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:06:35.113447 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:06:35.113447 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:06:35.113447 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:06:35.113447 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:06:35.113447 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:06:35.113447 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:06:35.113447 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:06:35.113447 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:06:35.113447 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:06:35.113447 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:06:35.113447 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:06:35.114530 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:06:35.114530 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:06:35.114530 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:06:35.114530 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:06:35.114530 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:06:35.114530 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:06:35.114530 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:06:35.114530 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:06:35.114530 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:06:35.114530 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:06:35.114530 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:06:35.114530 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:06:35.114530 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:06:35.114530 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:06:35.114530 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:06:35.114530 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:06:35.114530 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:06:35.115829 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:06:35.115829 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:06:35.115829 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:06:35.115829 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:06:35.115829 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 617/1251] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:06:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:06:35.120521 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570e6304-c4e2-4a76-ae80-36367d0a1dd8 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:35.221233 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570e6304-c4e2-4a76-ae80-36367d0a1dd8 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:35.255767 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aa0d8b-1731-4864-a598-084918fa4fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:35.256388 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aa0d8b-1731-4864-a598-084918fa4fd4 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:35.256670 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aa0d8b-1731-4864-a598-084918fa4fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:35.256945 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aa0d8b-1731-4864-a598-084918fa4fd4 None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:35.257628 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aa0d8b-1731-4864-a598-084918fa4fd4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:06:35.257965 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aa0d8b-1731-4864-a598-084918fa4fd4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:35.259338 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aa0d8b-1731-4864-a598-084918fa4fd4 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:35.270887 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 635/1252] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:06:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:06:35.276957 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fd92a3-3e30-4780-8f23-1f2dd74dabc4 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:35.383678 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fd92a3-3e30-4780-8f23-1f2dd74dabc4 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:35.408974 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:35.416684 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9809f5-3788-48a3-964f-08e48f103268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:35.420148 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9809f5-3788-48a3-964f-08e48f103268 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:35.420148 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9809f5-3788-48a3-964f-08e48f103268 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:35.420148 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9809f5-3788-48a3-964f-08e48f103268 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:35.425087 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9809f5-3788-48a3-964f-08e48f103268 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:35.426565 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9809f5-3788-48a3-964f-08e48f103268 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:35.437580 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-5f9809f5-3788-48a3-964f-08e48f103268 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 04 21:06:35.437580 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:06:35.437580 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:06:35.437580 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:06:35.437580 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:06:35.437580 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:06:35.437580 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:06:35.437580 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:06:35.437580 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:06:35.437580 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:06:35.437580 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:06:35.437580 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:06:35.437580 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 21:06:35.437580 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 21:06:35.437580 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 21:06:35.437580 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 21:06:35.438595 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:06:35.438595 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:06:35.438595 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 21:06:35.438595 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:06:35.438595 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 21:06:35.438595 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:06:35.438595 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:06:35.438595 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:06:35.438595 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:06:35.438595 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:06:35.438595 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:06:35.438595 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:06:35.438595 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:06:35.438595 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:06:35.438595 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:06:35.438595 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:06:35.438595 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:06:35.438595 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:06:35.440106 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:06:35.440106 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:06:35.440106 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 21:06:35.440106 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 21:06:35.440106 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 21:06:35.440106 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 21:06:35.440106 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 21:06:35.440106 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 21:06:35.440106 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 04 21:06:35.440106 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:06:35.440106 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 618/1253] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:06:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:06:35.445207 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aa0d8b-1731-4864-a598-084918fa4fd4 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:35.499391 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:35.544917 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aa0d8b-1731-4864-a598-084918fa4fd4 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:35.579954 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6a7adf-e238-4dce-84e1-88a77472f25e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:35.580576 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6a7adf-e238-4dce-84e1-88a77472f25e None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:35.580908 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6a7adf-e238-4dce-84e1-88a77472f25e None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:35.581783 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6a7adf-e238-4dce-84e1-88a77472f25e None admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:35.582587 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6a7adf-e238-4dce-84e1-88a77472f25e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:06:35.582892 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6a7adf-e238-4dce-84e1-88a77472f25e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:35.584363 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6a7adf-e238-4dce-84e1-88a77472f25e None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:35.611738 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 636/1254] 10.222.0.25 () {56 vars in 1143 bytes} [Thu Aug 4 21:06:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:06:35.618268 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9809f5-3788-48a3-964f-08e48f103268 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:35.716228 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9809f5-3788-48a3-964f-08e48f103268 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:35.748677 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115addd3-8b21-48ce-8bc1-4fdb3c589b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:35.749400 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115addd3-8b21-48ce-8bc1-4fdb3c589b2c None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:35.749683 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115addd3-8b21-48ce-8bc1-4fdb3c589b2c None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:35.749960 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115addd3-8b21-48ce-8bc1-4fdb3c589b2c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:35.804226 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:35.816518 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115addd3-8b21-48ce-8bc1-4fdb3c589b2c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:35.817982 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115addd3-8b21-48ce-8bc1-4fdb3c589b2c None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:35.911475 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 619/1255] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Aug 4 21:06:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 21:06:35.917724 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6a7adf-e238-4dce-84e1-88a77472f25e None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:36.019132 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6a7adf-e238-4dce-84e1-88a77472f25e None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:36.032175 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:36.054160 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4708d1f-a5af-44d1-8fe7-eeab44896439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:36.054796 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4708d1f-a5af-44d1-8fe7-eeab44896439 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:36.055078 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4708d1f-a5af-44d1-8fe7-eeab44896439 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:36.055375 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4708d1f-a5af-44d1-8fe7-eeab44896439 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:36.064984 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4708d1f-a5af-44d1-8fe7-eeab44896439 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:36.066444 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4708d1f-a5af-44d1-8fe7-eeab44896439 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:36.080715 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-b4708d1f-a5af-44d1-8fe7-eeab44896439 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:06:36.080715 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:06:36.080715 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:06:36.080715 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:06:36.080715 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:06:36.080715 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:06:36.080715 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:06:36.080715 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:06:36.080715 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:06:36.080715 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:06:36.080715 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:06:36.080715 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:06:36.080715 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:06:36.080715 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:06:36.080715 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:06:36.082353 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:06:36.082353 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:06:36.082353 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:06:36.082353 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:06:36.082353 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:06:36.082353 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:06:36.082353 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:06:36.082353 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:06:36.082353 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:06:36.082353 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:06:36.082353 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:06:36.082353 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:06:36.082353 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:06:36.082353 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:06:36.082353 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:06:36.082353 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:06:36.082353 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:06:36.083548 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:06:36.083548 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:06:36.083548 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:06:36.083548 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:06:36.083548 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 637/1256] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:06:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:06:36.087912 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115addd3-8b21-48ce-8bc1-4fdb3c589b2c None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:36.172645 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:36.185555 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115addd3-8b21-48ce-8bc1-4fdb3c589b2c None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:36.218524 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07794c51-1d86-41e5-9573-94dbe78fb164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:36.219143 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07794c51-1d86-41e5-9573-94dbe78fb164 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:36.219462 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07794c51-1d86-41e5-9573-94dbe78fb164 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:36.219737 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07794c51-1d86-41e5-9573-94dbe78fb164 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:36.220431 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07794c51-1d86-41e5-9573-94dbe78fb164 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:06:36.220739 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07794c51-1d86-41e5-9573-94dbe78fb164 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:36.222640 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07794c51-1d86-41e5-9573-94dbe78fb164 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:36.233844 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 620/1257] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:06:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:06:36.240316 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4708d1f-a5af-44d1-8fe7-eeab44896439 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:36.338898 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4708d1f-a5af-44d1-8fe7-eeab44896439 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:36.371431 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304e8478-9ddc-4c26-bdb1-58377fa1e1e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:36.372083 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304e8478-9ddc-4c26-bdb1-58377fa1e1e4 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:36.372365 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304e8478-9ddc-4c26-bdb1-58377fa1e1e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:36.372643 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304e8478-9ddc-4c26-bdb1-58377fa1e1e4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:36.379358 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304e8478-9ddc-4c26-bdb1-58377fa1e1e4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:36.380962 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304e8478-9ddc-4c26-bdb1-58377fa1e1e4 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:36.392091 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-304e8478-9ddc-4c26-bdb1-58377fa1e1e4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:06:36.392091 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:06:36.392091 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:06:36.392091 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:06:36.392091 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:06:36.392091 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:06:36.392091 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:06:36.392091 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:06:36.392091 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:06:36.392091 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:06:36.392091 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:06:36.392091 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:06:36.392091 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 21:06:36.392091 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 21:06:36.392091 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 21:06:36.393115 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 21:06:36.393115 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:06:36.393115 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:06:36.393115 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:06:36.393115 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:06:36.393115 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:06:36.393115 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:06:36.393115 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:06:36.393115 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:06:36.393115 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:06:36.393115 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:06:36.393115 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:06:36.393115 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:06:36.393115 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:06:36.393115 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:06:36.393115 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:06:36.393115 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:06:36.394741 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 21:06:36.394741 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 21:06:36.394741 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 21:06:36.394741 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 21:06:36.394741 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 21:06:36.394741 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 21:06:36.394741 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:06:36.394741 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:06:36.394741 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 638/1258] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Aug 4 21:06:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:06:36.399744 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07794c51-1d86-41e5-9573-94dbe78fb164 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:36.499004 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07794c51-1d86-41e5-9573-94dbe78fb164 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:36.532337 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61494c14-878f-4b97-ab90-005474c7d415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:36.532947 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61494c14-878f-4b97-ab90-005474c7d415 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:36.533229 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61494c14-878f-4b97-ab90-005474c7d415 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:36.533502 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61494c14-878f-4b97-ab90-005474c7d415 None admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:36.534861 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61494c14-878f-4b97-ab90-005474c7d415 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:06:36.535164 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61494c14-878f-4b97-ab90-005474c7d415 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:36.536628 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61494c14-878f-4b97-ab90-005474c7d415 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:36.548715 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 621/1259] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Aug 4 21:06:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:06:36.555062 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304e8478-9ddc-4c26-bdb1-58377fa1e1e4 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:36.580007 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:36.655020 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304e8478-9ddc-4c26-bdb1-58377fa1e1e4 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:36.691221 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172456d7-3cb7-4e4e-836c-fd6dd897cce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:36.691858 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172456d7-3cb7-4e4e-836c-fd6dd897cce3 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:36.692138 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172456d7-3cb7-4e4e-836c-fd6dd897cce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:36.692407 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172456d7-3cb7-4e4e-836c-fd6dd897cce3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:36.693180 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172456d7-3cb7-4e4e-836c-fd6dd897cce3 None admin] RBAC: Adding query filter params (role.id=a432dd4735944ecfab2fa5ce1705d356, scope.project.id=17dc3d39e2c048ca9070588f5717a578, user.id=67d44a0098ab4c36b14b36e9e59ae75c) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:06:36.693486 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172456d7-3cb7-4e4e-836c-fd6dd897cce3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:36.694920 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172456d7-3cb7-4e4e-836c-fd6dd897cce3 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:36.700337 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 639/1260] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Aug 4 21:06:36 2022] GET /identity/v3/role_assignments?user.id=67d44a0098ab4c36b14b36e9e59ae75c&scope.project.id=17dc3d39e2c048ca9070588f5717a578&role.id=a432dd4735944ecfab2fa5ce1705d356 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:06:38.732812 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e13e7c-67c4-4b28-bb6b-0ab780b4f4b8 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:38.733345 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e13e7c-67c4-4b28-bb6b-0ab780b4f4b8 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:38.733345 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e13e7c-67c4-4b28-bb6b-0ab780b4f4b8 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:38.734720 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 622/1261] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:06:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:06:38.740650 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5920d652-63e6-4228-8dc8-1a94728a62e1 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:38.741082 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5920d652-63e6-4228-8dc8-1a94728a62e1 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:38.741206 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5920d652-63e6-4228-8dc8-1a94728a62e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:38.759708 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-5920d652-63e6-4228-8dc8-1a94728a62e1 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:06:38.760926 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-5920d652-63e6-4228-8dc8-1a94728a62e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:38.826335 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 640/1262] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:06:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:06:38.848452 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e13e7c-67c4-4b28-bb6b-0ab780b4f4b8 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:38.859555 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:38.949670 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e13e7c-67c4-4b28-bb6b-0ab780b4f4b8 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:38.983535 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cf2b5a-4d1d-43be-a759-8bf6a889c6cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:38.984093 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cf2b5a-4d1d-43be-a759-8bf6a889c6cc None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:38.984374 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cf2b5a-4d1d-43be-a759-8bf6a889c6cc None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:38.984645 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cf2b5a-4d1d-43be-a759-8bf6a889c6cc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:38.995272 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cf2b5a-4d1d-43be-a759-8bf6a889c6cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:06:39.000256 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-b0cf2b5a-4d1d-43be-a759-8bf6a889c6cc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:06:39.000256 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:06:39.000256 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:06:39.000256 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:06:39.000256 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:06:39.000256 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:06:39.000256 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:06:39.000256 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:06:39.000256 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:06:39.000256 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:06:39.000256 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:06:39.000256 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:06:39.000256 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:06:39.000256 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:06:39.000256 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:06:39.000256 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:06:39.001314 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:06:39.001314 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:06:39.001314 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:06:39.001314 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:06:39.001314 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:06:39.001314 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:06:39.001314 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:06:39.001314 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:06:39.001314 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:06:39.001314 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:06:39.001314 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:06:39.001314 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:06:39.001314 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:06:39.001314 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:06:39.001314 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:06:39.001314 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:06:39.002170 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:06:39.002170 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:06:39.002170 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:06:39.002170 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:06:39.002170 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:06:39.002170 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:06:39.002170 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:06:39.002170 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:06:39.002170 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:06:39.002170 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:06:39.002170 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:06:39.002170 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:06:39.002170 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 623/1263] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Aug 4 21:06:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:06:39.007964 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5920d652-63e6-4228-8dc8-1a94728a62e1 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:39.026181 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:39.105825 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5920d652-63e6-4228-8dc8-1a94728a62e1 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:39.140359 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e199743-79a6-406d-a2ee-7e7f9b99f86a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:39.140980 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e199743-79a6-406d-a2ee-7e7f9b99f86a None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:39.141266 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e199743-79a6-406d-a2ee-7e7f9b99f86a None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:39.141537 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e199743-79a6-406d-a2ee-7e7f9b99f86a None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:39.142252 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e199743-79a6-406d-a2ee-7e7f9b99f86a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:06:39.142558 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e199743-79a6-406d-a2ee-7e7f9b99f86a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:39.144554 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e199743-79a6-406d-a2ee-7e7f9b99f86a None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:39.152496 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 641/1264] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Aug 4 21:06:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:06:39.158523 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cf2b5a-4d1d-43be-a759-8bf6a889c6cc None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:39.227802 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:39.261293 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cf2b5a-4d1d-43be-a759-8bf6a889c6cc None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:39.294142 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373298d6-cf6b-4c45-be13-4080d38924c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:39.294768 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373298d6-cf6b-4c45-be13-4080d38924c3 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:39.295048 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373298d6-cf6b-4c45-be13-4080d38924c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:39.295381 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373298d6-cf6b-4c45-be13-4080d38924c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:39.306850 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373298d6-cf6b-4c45-be13-4080d38924c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:39.308367 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373298d6-cf6b-4c45-be13-4080d38924c3 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:39.322610 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-373298d6-cf6b-4c45-be13-4080d38924c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:06:39.322610 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:06:39.322610 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:06:39.322610 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:06:39.322610 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:06:39.322610 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:06:39.322610 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:06:39.322610 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:06:39.322610 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:06:39.322610 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:06:39.322610 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:06:39.322610 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:06:39.322610 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:06:39.322610 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:06:39.322610 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:06:39.323738 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:06:39.323738 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:06:39.323738 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:06:39.323738 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:06:39.323738 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:06:39.323738 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:06:39.323738 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:06:39.323738 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:06:39.323738 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:06:39.323738 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:06:39.323738 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:06:39.323738 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:06:39.323738 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:06:39.323738 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:06:39.323738 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:06:39.323738 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:06:39.323738 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:06:39.324697 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:06:39.324697 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:06:39.324697 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:06:39.324697 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:06:39.324697 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 624/1265] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:06:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:06:39.329854 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e199743-79a6-406d-a2ee-7e7f9b99f86a None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:39.414069 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:39.432497 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e199743-79a6-406d-a2ee-7e7f9b99f86a None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:39.465594 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b82638e-0b83-46c3-976f-23c4c88ea1eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:39.466217 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b82638e-0b83-46c3-976f-23c4c88ea1eb None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:39.466503 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b82638e-0b83-46c3-976f-23c4c88ea1eb None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:39.466784 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b82638e-0b83-46c3-976f-23c4c88ea1eb None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:39.467503 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b82638e-0b83-46c3-976f-23c4c88ea1eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:06:39.467813 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b82638e-0b83-46c3-976f-23c4c88ea1eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:39.469185 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b82638e-0b83-46c3-976f-23c4c88ea1eb None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:39.480726 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 642/1266] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:06:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:06:39.486632 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373298d6-cf6b-4c45-be13-4080d38924c3 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:39.585327 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373298d6-cf6b-4c45-be13-4080d38924c3 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:39.616504 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:39.629788 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284107f7-db99-4098-a71d-f5e7e617a012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:39.630486 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284107f7-db99-4098-a71d-f5e7e617a012 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:39.630772 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284107f7-db99-4098-a71d-f5e7e617a012 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:39.631045 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284107f7-db99-4098-a71d-f5e7e617a012 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:39.651339 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284107f7-db99-4098-a71d-f5e7e617a012 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:39.652884 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284107f7-db99-4098-a71d-f5e7e617a012 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:39.670946 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-284107f7-db99-4098-a71d-f5e7e617a012 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 04 21:06:39.670946 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:06:39.670946 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:06:39.670946 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:06:39.670946 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:06:39.670946 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:06:39.670946 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:06:39.670946 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:06:39.670946 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:06:39.670946 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:06:39.670946 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:06:39.670946 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:06:39.670946 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 21:06:39.670946 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 21:06:39.670946 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 21:06:39.670946 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 21:06:39.672103 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:06:39.672103 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:06:39.672103 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 21:06:39.672103 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:06:39.672103 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 21:06:39.672103 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:06:39.672103 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:06:39.672103 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:06:39.672103 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:06:39.672103 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:06:39.672103 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:06:39.672103 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:06:39.672103 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:06:39.672103 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:06:39.672103 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:06:39.672103 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:06:39.672103 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:06:39.672103 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:06:39.673136 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:06:39.673136 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:06:39.673136 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 21:06:39.673136 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 21:06:39.673136 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 21:06:39.673136 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 21:06:39.673136 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 21:06:39.673136 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 21:06:39.673136 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 04 21:06:39.673136 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:06:39.673136 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 625/1267] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:06:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:06:39.678135 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b82638e-0b83-46c3-976f-23c4c88ea1eb None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:39.783460 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b82638e-0b83-46c3-976f-23c4c88ea1eb None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:39.816255 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f27933d-0d2c-4673-8b01-1d5c9ca30365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:39.816866 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f27933d-0d2c-4673-8b01-1d5c9ca30365 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:39.817142 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f27933d-0d2c-4673-8b01-1d5c9ca30365 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:39.817408 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f27933d-0d2c-4673-8b01-1d5c9ca30365 None admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:39.818206 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f27933d-0d2c-4673-8b01-1d5c9ca30365 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:06:39.818543 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f27933d-0d2c-4673-8b01-1d5c9ca30365 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:39.820000 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f27933d-0d2c-4673-8b01-1d5c9ca30365 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:39.829043 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:39.837554 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 643/1268] 10.222.0.25 () {56 vars in 1143 bytes} [Thu Aug 4 21:06:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:06:39.843410 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284107f7-db99-4098-a71d-f5e7e617a012 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:39.939800 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284107f7-db99-4098-a71d-f5e7e617a012 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:39.972712 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a043beb0-7652-4ed3-962c-022bc05da450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:39.973353 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a043beb0-7652-4ed3-962c-022bc05da450 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:39.973639 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a043beb0-7652-4ed3-962c-022bc05da450 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:39.973917 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a043beb0-7652-4ed3-962c-022bc05da450 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:39.984058 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a043beb0-7652-4ed3-962c-022bc05da450 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:39.992807 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a043beb0-7652-4ed3-962c-022bc05da450 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:39.999686 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-a043beb0-7652-4ed3-962c-022bc05da450 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:06:39.999686 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:06:39.999686 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:06:39.999686 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:06:39.999686 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:06:39.999686 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:06:39.999686 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:06:39.999686 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:06:39.999686 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:06:39.999686 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:06:39.999686 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:06:39.999686 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:06:39.999686 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:06:39.999686 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:06:39.999686 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:06:40.000792 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:06:40.000792 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:06:40.000792 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:06:40.000792 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:06:40.000792 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:06:40.000792 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:06:40.000792 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:06:40.000792 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:06:40.000792 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:06:40.000792 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:06:40.000792 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:06:40.000792 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:06:40.000792 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:06:40.000792 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:06:40.000792 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:06:40.000792 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:06:40.000792 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:06:40.001777 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:06:40.001777 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:06:40.001777 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:06:40.001777 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:06:40.001777 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 626/1269] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:06:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:06:40.007084 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f27933d-0d2c-4673-8b01-1d5c9ca30365 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:40.114258 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f27933d-0d2c-4673-8b01-1d5c9ca30365 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:40.148000 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7ea725-5b9e-442c-bac1-1b7cfe92548d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:40.148614 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7ea725-5b9e-442c-bac1-1b7cfe92548d None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:40.148894 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7ea725-5b9e-442c-bac1-1b7cfe92548d None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:40.149162 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7ea725-5b9e-442c-bac1-1b7cfe92548d None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:40.149873 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7ea725-5b9e-442c-bac1-1b7cfe92548d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:06:40.150171 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7ea725-5b9e-442c-bac1-1b7cfe92548d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:40.151553 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7ea725-5b9e-442c-bac1-1b7cfe92548d None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:40.162508 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 644/1270] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:06:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:06:40.168482 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a043beb0-7652-4ed3-962c-022bc05da450 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:40.200632 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:40.269251 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a043beb0-7652-4ed3-962c-022bc05da450 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:40.303195 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417fc1b1-dd31-4516-8f7e-be1685ef66c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:40.303875 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417fc1b1-dd31-4516-8f7e-be1685ef66c1 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:40.304161 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417fc1b1-dd31-4516-8f7e-be1685ef66c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:40.304434 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417fc1b1-dd31-4516-8f7e-be1685ef66c1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:40.311138 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417fc1b1-dd31-4516-8f7e-be1685ef66c1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:40.312634 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417fc1b1-dd31-4516-8f7e-be1685ef66c1 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:40.323810 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-417fc1b1-dd31-4516-8f7e-be1685ef66c1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:06:40.323810 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:06:40.323810 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:06:40.323810 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:06:40.323810 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:06:40.323810 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:06:40.323810 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:06:40.323810 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:06:40.323810 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:06:40.323810 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:06:40.323810 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:06:40.323810 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:06:40.323810 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 21:06:40.323810 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 21:06:40.323810 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 21:06:40.324904 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 21:06:40.324904 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:06:40.324904 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:06:40.324904 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:06:40.324904 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:06:40.324904 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:06:40.324904 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:06:40.324904 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:06:40.324904 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:06:40.324904 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:06:40.324904 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:06:40.324904 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:06:40.324904 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:06:40.324904 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:06:40.324904 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:06:40.324904 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:06:40.324904 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:06:40.325940 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 21:06:40.325940 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 21:06:40.325940 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 21:06:40.325940 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 21:06:40.325940 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 21:06:40.325940 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 21:06:40.325940 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:06:40.325940 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:06:40.325940 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 627/1271] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Aug 4 21:06:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:06:40.331383 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7ea725-5b9e-442c-bac1-1b7cfe92548d None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:40.388626 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:40.430377 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7ea725-5b9e-442c-bac1-1b7cfe92548d None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:40.463591 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df734a24-0e0b-4a13-ab59-02b9f0c99bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:40.464215 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df734a24-0e0b-4a13-ab59-02b9f0c99bda None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:40.464499 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df734a24-0e0b-4a13-ab59-02b9f0c99bda None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:40.464778 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df734a24-0e0b-4a13-ab59-02b9f0c99bda None admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:40.465527 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df734a24-0e0b-4a13-ab59-02b9f0c99bda None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:06:40.465837 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df734a24-0e0b-4a13-ab59-02b9f0c99bda None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:40.467282 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df734a24-0e0b-4a13-ab59-02b9f0c99bda None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:40.479627 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 645/1272] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Aug 4 21:06:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:06:40.485654 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417fc1b1-dd31-4516-8f7e-be1685ef66c1 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:40.576760 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:40.585661 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417fc1b1-dd31-4516-8f7e-be1685ef66c1 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:40.617694 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae59be3a-87bd-4970-a3b0-216cad387d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:40.618355 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae59be3a-87bd-4970-a3b0-216cad387d8c None admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:40.618637 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae59be3a-87bd-4970-a3b0-216cad387d8c None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:40.618911 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae59be3a-87bd-4970-a3b0-216cad387d8c None admin] PATH_INFO: `/v3/projects/17dc3d39e2c048ca9070588f5717a578/users/67d44a0098ab4c36b14b36e9e59ae75c/roles/a432dd4735944ecfab2fa5ce1705d356` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:40.620844 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae59be3a-87bd-4970-a3b0-216cad387d8c None admin] RBAC: Authorizing `identity:create_grant(project_id=17dc3d39e2c048ca9070588f5717a578, user_id=67d44a0098ab4c36b14b36e9e59ae75c, role_id=a432dd4735944ecfab2fa5ce1705d356)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:40.622280 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae59be3a-87bd-4970-a3b0-216cad387d8c None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:40.742770 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 628/1273] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Aug 4 21:06:40 2022] PUT /identity/v3/projects/17dc3d39e2c048ca9070588f5717a578/users/67d44a0098ab4c36b14b36e9e59ae75c/roles/a432dd4735944ecfab2fa5ce1705d356 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:06:42.732870 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b3e5b2-e100-4a11-a51b-115d3c620942 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:42.733443 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b3e5b2-e100-4a11-a51b-115d3c620942 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:42.733443 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b3e5b2-e100-4a11-a51b-115d3c620942 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:42.734756 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 646/1274] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:06:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:06:42.740273 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cf5d77-d9f9-4f29-9b5b-794b3f32b43b None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:42.740676 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cf5d77-d9f9-4f29-9b5b-794b3f32b43b None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:42.740834 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cf5d77-d9f9-4f29-9b5b-794b3f32b43b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:42.759694 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-f4cf5d77-d9f9-4f29-9b5b-794b3f32b43b None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:06:42.760884 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-f4cf5d77-d9f9-4f29-9b5b-794b3f32b43b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:42.826567 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 629/1275] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:06:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:06:42.850530 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b3e5b2-e100-4a11-a51b-115d3c620942 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:42.950027 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b3e5b2-e100-4a11-a51b-115d3c620942 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:42.967786 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:42.983909 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3f90f0-b847-4fd5-9f84-8868d4216cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:42.984488 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3f90f0-b847-4fd5-9f84-8868d4216cc4 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:42.984771 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3f90f0-b847-4fd5-9f84-8868d4216cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:42.985047 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3f90f0-b847-4fd5-9f84-8868d4216cc4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:42.995903 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3f90f0-b847-4fd5-9f84-8868d4216cc4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:06:43.001262 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-2a3f90f0-b847-4fd5-9f84-8868d4216cc4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:06:43.001262 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:06:43.001262 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:06:43.001262 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:06:43.001262 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:06:43.001262 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:06:43.001262 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:06:43.001262 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:06:43.001262 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:06:43.001262 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:06:43.001262 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:06:43.001262 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:06:43.001262 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:06:43.001262 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:06:43.001262 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:06:43.001262 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:06:43.002408 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:06:43.002408 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:06:43.002408 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:06:43.002408 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:06:43.002408 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:06:43.002408 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:06:43.002408 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:06:43.002408 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:06:43.002408 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:06:43.002408 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:06:43.002408 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:06:43.002408 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:06:43.002408 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:06:43.002408 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:06:43.002408 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:06:43.002408 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:06:43.003841 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:06:43.003841 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:06:43.003841 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:06:43.003841 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:06:43.003841 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:06:43.003841 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:06:43.003841 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:06:43.003841 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:06:43.003841 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:06:43.003841 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:06:43.003841 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:06:43.003841 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:06:43.003841 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 647/1276] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Aug 4 21:06:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:06:43.009105 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cf5d77-d9f9-4f29-9b5b-794b3f32b43b None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:43.087631 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:43.105311 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cf5d77-d9f9-4f29-9b5b-794b3f32b43b None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:43.138619 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7336f447-a819-4000-a038-2713ecb771f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:43.139233 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7336f447-a819-4000-a038-2713ecb771f5 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:43.139541 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7336f447-a819-4000-a038-2713ecb771f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:43.139824 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7336f447-a819-4000-a038-2713ecb771f5 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:43.140539 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7336f447-a819-4000-a038-2713ecb771f5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:06:43.140842 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7336f447-a819-4000-a038-2713ecb771f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:43.142232 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7336f447-a819-4000-a038-2713ecb771f5 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:43.150270 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 630/1277] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Aug 4 21:06:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:06:43.156166 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3f90f0-b847-4fd5-9f84-8868d4216cc4 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:43.258846 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3f90f0-b847-4fd5-9f84-8868d4216cc4 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:43.291479 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40921795-6268-49c2-a17f-e2f550f268af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:43.292112 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40921795-6268-49c2-a17f-e2f550f268af None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:43.292392 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40921795-6268-49c2-a17f-e2f550f268af None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:43.292661 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40921795-6268-49c2-a17f-e2f550f268af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:43.302958 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40921795-6268-49c2-a17f-e2f550f268af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:43.304422 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40921795-6268-49c2-a17f-e2f550f268af None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:43.319171 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-40921795-6268-49c2-a17f-e2f550f268af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:06:43.319171 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:06:43.319171 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:06:43.319171 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:06:43.319171 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:06:43.319171 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:06:43.319171 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:06:43.319171 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:06:43.319171 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:06:43.319171 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:06:43.319171 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:06:43.319171 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:06:43.319171 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:06:43.319171 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:06:43.319171 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:06:43.320108 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:06:43.320108 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:06:43.320108 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:06:43.320108 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:06:43.320108 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:06:43.320108 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:06:43.320108 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:06:43.320108 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:06:43.320108 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:06:43.320108 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:06:43.320108 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:06:43.320108 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:06:43.320108 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:06:43.320108 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:06:43.320108 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:06:43.320108 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:06:43.320108 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:06:43.321144 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:06:43.321144 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:06:43.321144 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:06:43.321144 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:06:43.321144 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 648/1278] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:06:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:06:43.326357 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7336f447-a819-4000-a038-2713ecb771f5 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:43.425793 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7336f447-a819-4000-a038-2713ecb771f5 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:43.459946 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3246e8a2-0eb1-4a39-8bc3-ca772452355a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:43.460561 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3246e8a2-0eb1-4a39-8bc3-ca772452355a None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:43.462008 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3246e8a2-0eb1-4a39-8bc3-ca772452355a None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:43.462008 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3246e8a2-0eb1-4a39-8bc3-ca772452355a None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:43.462008 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3246e8a2-0eb1-4a39-8bc3-ca772452355a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:06:43.462008 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3246e8a2-0eb1-4a39-8bc3-ca772452355a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:43.463416 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3246e8a2-0eb1-4a39-8bc3-ca772452355a None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:43.475485 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 631/1279] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:06:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:06:43.481224 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40921795-6268-49c2-a17f-e2f550f268af None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:43.511736 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:43.579090 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40921795-6268-49c2-a17f-e2f550f268af None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:43.613798 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e30003-aa4e-4cdf-b775-7cbd8a8ae223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:43.614445 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e30003-aa4e-4cdf-b775-7cbd8a8ae223 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:43.614725 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e30003-aa4e-4cdf-b775-7cbd8a8ae223 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:43.614995 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e30003-aa4e-4cdf-b775-7cbd8a8ae223 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:43.622009 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e30003-aa4e-4cdf-b775-7cbd8a8ae223 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:43.623471 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e30003-aa4e-4cdf-b775-7cbd8a8ae223 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:43.639663 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-77e30003-aa4e-4cdf-b775-7cbd8a8ae223 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 04 21:06:43.639663 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:06:43.639663 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:06:43.639663 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:06:43.639663 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:06:43.639663 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:06:43.639663 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:06:43.639663 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:06:43.639663 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:06:43.639663 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:06:43.639663 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:06:43.639663 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:06:43.639663 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 21:06:43.639663 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 21:06:43.639663 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 21:06:43.639663 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 21:06:43.640834 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:06:43.640834 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:06:43.640834 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 21:06:43.640834 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:06:43.640834 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 21:06:43.640834 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:06:43.640834 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:06:43.640834 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:06:43.640834 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:06:43.640834 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:06:43.640834 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:06:43.640834 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:06:43.640834 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:06:43.640834 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:06:43.640834 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:06:43.640834 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:06:43.640834 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:06:43.640834 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:06:43.642014 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:06:43.642014 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:06:43.642014 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 21:06:43.642014 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 21:06:43.642014 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 21:06:43.642014 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 21:06:43.642014 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 21:06:43.642014 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 21:06:43.642014 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 04 21:06:43.642014 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:06:43.642014 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 649/1280] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:06:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:06:43.647655 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3246e8a2-0eb1-4a39-8bc3-ca772452355a None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:43.651548 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:43.749781 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3246e8a2-0eb1-4a39-8bc3-ca772452355a None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:43.784037 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612fc454-0b8d-4db8-96f8-b2a762c854d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:43.784605 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612fc454-0b8d-4db8-96f8-b2a762c854d0 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:43.784883 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612fc454-0b8d-4db8-96f8-b2a762c854d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:43.785158 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612fc454-0b8d-4db8-96f8-b2a762c854d0 None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:43.785944 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612fc454-0b8d-4db8-96f8-b2a762c854d0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:06:43.786252 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612fc454-0b8d-4db8-96f8-b2a762c854d0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:43.787732 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612fc454-0b8d-4db8-96f8-b2a762c854d0 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:43.805386 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 632/1281] 10.222.0.25 () {56 vars in 1143 bytes} [Thu Aug 4 21:06:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:06:43.811141 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e30003-aa4e-4cdf-b775-7cbd8a8ae223 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:43.900797 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:43.910304 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e30003-aa4e-4cdf-b775-7cbd8a8ae223 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:43.944854 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03db13d4-242b-4142-80a2-92fb808651ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:43.945472 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03db13d4-242b-4142-80a2-92fb808651ed None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:43.945755 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03db13d4-242b-4142-80a2-92fb808651ed None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:43.946018 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03db13d4-242b-4142-80a2-92fb808651ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:44.011702 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03db13d4-242b-4142-80a2-92fb808651ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:44.013255 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03db13d4-242b-4142-80a2-92fb808651ed None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:44.102277 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:44.111893 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 650/1282] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Aug 4 21:06:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 21:06:44.118470 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612fc454-0b8d-4db8-96f8-b2a762c854d0 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:44.178816 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:44.215872 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612fc454-0b8d-4db8-96f8-b2a762c854d0 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:44.251703 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7381a43d-dbe0-465b-b933-769b2d9d7662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:44.252340 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7381a43d-dbe0-465b-b933-769b2d9d7662 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:44.252618 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7381a43d-dbe0-465b-b933-769b2d9d7662 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:44.252934 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7381a43d-dbe0-465b-b933-769b2d9d7662 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:44.263862 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7381a43d-dbe0-465b-b933-769b2d9d7662 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:44.265320 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7381a43d-dbe0-465b-b933-769b2d9d7662 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:44.279895 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-7381a43d-dbe0-465b-b933-769b2d9d7662 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:06:44.279895 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:06:44.279895 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:06:44.279895 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:06:44.279895 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:06:44.279895 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:06:44.279895 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:06:44.279895 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:06:44.279895 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:06:44.279895 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:06:44.279895 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:06:44.279895 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:06:44.279895 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:06:44.279895 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:06:44.279895 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:06:44.280956 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:06:44.280956 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:06:44.280956 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:06:44.280956 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:06:44.280956 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:06:44.280956 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:06:44.280956 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:06:44.280956 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:06:44.280956 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:06:44.280956 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:06:44.280956 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:06:44.280956 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:06:44.280956 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:06:44.280956 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:06:44.280956 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:06:44.280956 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:06:44.280956 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:06:44.282063 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:06:44.282063 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:06:44.282063 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:06:44.282063 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:06:44.282063 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 633/1283] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:06:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:06:44.287368 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03db13d4-242b-4142-80a2-92fb808651ed None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:44.384562 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03db13d4-242b-4142-80a2-92fb808651ed None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:44.419937 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577e2159-a40f-4507-a483-8d7414911c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:44.420566 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577e2159-a40f-4507-a483-8d7414911c9d None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:44.420983 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577e2159-a40f-4507-a483-8d7414911c9d None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:44.421324 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577e2159-a40f-4507-a483-8d7414911c9d None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:44.422032 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577e2159-a40f-4507-a483-8d7414911c9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:06:44.422337 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577e2159-a40f-4507-a483-8d7414911c9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:44.423760 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577e2159-a40f-4507-a483-8d7414911c9d None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:44.438653 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 651/1284] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Aug 4 21:06:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:06:44.444607 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7381a43d-dbe0-465b-b933-769b2d9d7662 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:44.547941 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7381a43d-dbe0-465b-b933-769b2d9d7662 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:44.563247 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:44.583495 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f8b093-5cae-44b2-a284-56b6df14c3f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:44.584163 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f8b093-5cae-44b2-a284-56b6df14c3f5 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:44.584503 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f8b093-5cae-44b2-a284-56b6df14c3f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:44.584720 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f8b093-5cae-44b2-a284-56b6df14c3f5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:44.591148 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f8b093-5cae-44b2-a284-56b6df14c3f5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:44.593752 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f8b093-5cae-44b2-a284-56b6df14c3f5 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:44.604309 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-85f8b093-5cae-44b2-a284-56b6df14c3f5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:06:44.604309 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:06:44.604309 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:06:44.604309 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:06:44.604309 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:06:44.604309 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:06:44.604309 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:06:44.604309 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:06:44.604309 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:06:44.604309 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:06:44.604309 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:06:44.604309 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:06:44.604309 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 21:06:44.604309 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 21:06:44.604309 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 21:06:44.605451 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 21:06:44.605451 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:06:44.605451 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:06:44.605451 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:06:44.605451 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:06:44.605451 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:06:44.605451 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:06:44.605451 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:06:44.605451 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Aug 04 21:06:44.605451 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:06:44.605451 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:06:44.605451 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:06:44.605451 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:06:44.605451 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:06:44.605451 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:06:44.605451 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:06:44.605451 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:06:44.606387 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 21:06:44.606387 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 21:06:44.606387 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 21:06:44.606387 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 21:06:44.606387 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 21:06:44.606387 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 21:06:44.606387 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:06:44.606387 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Aug 04 21:06:44.606387 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 634/1285] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Aug 4 21:06:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:06:44.611462 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577e2159-a40f-4507-a483-8d7414911c9d None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:44.662879 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:44.711596 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577e2159-a40f-4507-a483-8d7414911c9d None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:44.747095 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02249d2-0173-49e1-9ac6-75dbe261f22c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:44.747789 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02249d2-0173-49e1-9ac6-75dbe261f22c None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:44.748046 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02249d2-0173-49e1-9ac6-75dbe261f22c None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:44.748319 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02249d2-0173-49e1-9ac6-75dbe261f22c None admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:44.749077 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02249d2-0173-49e1-9ac6-75dbe261f22c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:06:44.749381 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02249d2-0173-49e1-9ac6-75dbe261f22c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:44.750817 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02249d2-0173-49e1-9ac6-75dbe261f22c None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:44.763834 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 652/1286] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Aug 4 21:06:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:06:44.770530 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f8b093-5cae-44b2-a284-56b6df14c3f5 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:44.869108 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f8b093-5cae-44b2-a284-56b6df14c3f5 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:44.902156 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711ce9f4-4ac3-4e3c-a01a-29c340953a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:44.902771 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711ce9f4-4ac3-4e3c-a01a-29c340953a2a None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:44.903057 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711ce9f4-4ac3-4e3c-a01a-29c340953a2a None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:44.903352 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711ce9f4-4ac3-4e3c-a01a-29c340953a2a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:44.904135 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711ce9f4-4ac3-4e3c-a01a-29c340953a2a None admin] RBAC: Adding query filter params (role.id=a432dd4735944ecfab2fa5ce1705d356, scope.project.id=17dc3d39e2c048ca9070588f5717a578, user.id=67d44a0098ab4c36b14b36e9e59ae75c) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:06:44.904445 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711ce9f4-4ac3-4e3c-a01a-29c340953a2a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:44.905979 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711ce9f4-4ac3-4e3c-a01a-29c340953a2a None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:44.912688 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 635/1287] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Aug 4 21:06:44 2022] GET /identity/v3/role_assignments?user.id=67d44a0098ab4c36b14b36e9e59ae75c&scope.project.id=17dc3d39e2c048ca9070588f5717a578&role.id=a432dd4735944ecfab2fa5ce1705d356 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:06:46.932128 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d632434-8fd8-4dcf-bbcd-87ed08746b40 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:46.932658 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d632434-8fd8-4dcf-bbcd-87ed08746b40 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:46.932658 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d632434-8fd8-4dcf-bbcd-87ed08746b40 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:46.934028 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 653/1288] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:06:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:06:46.939254 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7978b65-5502-49d0-a802-5b3d14f47b7e None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:46.939702 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7978b65-5502-49d0-a802-5b3d14f47b7e None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:46.939840 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7978b65-5502-49d0-a802-5b3d14f47b7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:46.958972 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-d7978b65-5502-49d0-a802-5b3d14f47b7e None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:06:46.960201 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-d7978b65-5502-49d0-a802-5b3d14f47b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:46.989011 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:47.025285 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 636/1289] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:06:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:06:47.047133 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d632434-8fd8-4dcf-bbcd-87ed08746b40 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:47.148908 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d632434-8fd8-4dcf-bbcd-87ed08746b40 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:47.160117 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:47.184311 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02898ef5-11d1-4a06-b354-e5ddd18b907f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:47.184921 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02898ef5-11d1-4a06-b354-e5ddd18b907f None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:47.185257 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02898ef5-11d1-4a06-b354-e5ddd18b907f None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:47.185541 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02898ef5-11d1-4a06-b354-e5ddd18b907f None admin] PATH_INFO: `/v3/services/placement` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:47.186178 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02898ef5-11d1-4a06-b354-e5ddd18b907f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:47.187598 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02898ef5-11d1-4a06-b354-e5ddd18b907f None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:47.197115 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-02898ef5-11d1-4a06-b354-e5ddd18b907f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 21:06:47.197115 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:06:47.197115 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 21:06:47.197115 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 21:06:47.197115 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 21:06:47.197115 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 21:06:47.197115 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 21:06:47.197115 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:06:47.197115 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 21:06:47.197115 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:06:47.197115 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 21:06:47.197115 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:06:47.197115 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:06:47.197115 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:06:47.197115 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:06:47.197115 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:06:47.197115 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:06:47.198298 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:06:47.198298 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:06:47.198298 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:06:47.198298 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:06:47.198298 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:06:47.198298 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:06:47.198298 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 21:06:47.198298 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 21:06:47.198298 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 21:06:47.198298 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 21:06:47.198298 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:06:47.198298 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:06:47.198298 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:06:47.198298 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:06:47.198298 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:06:47.198298 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:06:47.199218 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:06:47.199218 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:06:47.199218 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:06:47.199218 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:06:47.199218 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:06:47.199218 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:06:47.199218 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:06:47.199218 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:06:47.199218 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:06:47.199218 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:06:47.199218 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 21:06:47.199218 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:06:47.199218 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 21:06:47.199218 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:06:47.199218 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 654/1290] 10.222.0.25 () {56 vars in 1117 bytes} [Thu Aug 4 21:06:47 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:06:47.204463 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7978b65-5502-49d0-a802-5b3d14f47b7e None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:47.306908 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7978b65-5502-49d0-a802-5b3d14f47b7e None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:47.340380 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c4b0d6-4e3d-4578-bd56-220e34f9945b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:47.340992 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c4b0d6-4e3d-4578-bd56-220e34f9945b None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:47.341269 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c4b0d6-4e3d-4578-bd56-220e34f9945b None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:47.341543 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c4b0d6-4e3d-4578-bd56-220e34f9945b None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:47.342267 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c4b0d6-4e3d-4578-bd56-220e34f9945b None admin] RBAC: Adding query filter params (name=placement) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:06:47.342563 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c4b0d6-4e3d-4578-bd56-220e34f9945b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:47.343982 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c4b0d6-4e3d-4578-bd56-220e34f9945b None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:47.348887 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 637/1291] 10.222.0.25 () {56 vars in 1116 bytes} [Thu Aug 4 21:06:47 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:06:47.354617 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02898ef5-11d1-4a06-b354-e5ddd18b907f None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:47.453444 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02898ef5-11d1-4a06-b354-e5ddd18b907f None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:47.486587 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f99158d-6dd4-4408-b74d-68f25e8d76f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:47.487400 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f99158d-6dd4-4408-b74d-68f25e8d76f8 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:47.487688 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f99158d-6dd4-4408-b74d-68f25e8d76f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:47.487963 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f99158d-6dd4-4408-b74d-68f25e8d76f8 None admin] PATH_INFO: `/v3/services` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:47.488634 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f99158d-6dd4-4408-b74d-68f25e8d76f8 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:06:47.488938 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f99158d-6dd4-4408-b74d-68f25e8d76f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:47.490308 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f99158d-6dd4-4408-b74d-68f25e8d76f8 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:47.494704 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 655/1292] 10.222.0.25 () {56 vars in 1116 bytes} [Thu Aug 4 21:06:47 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:06:49.529219 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2340b656-a397-4713-9947-3a318f776a15 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:49.529732 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2340b656-a397-4713-9947-3a318f776a15 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:49.529732 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2340b656-a397-4713-9947-3a318f776a15 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:49.531048 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 638/1293] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:06:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:06:49.536785 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc0009c-284d-4541-891e-4514de1ed1eb None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:49.537162 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc0009c-284d-4541-891e-4514de1ed1eb None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:49.537338 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc0009c-284d-4541-891e-4514de1ed1eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:49.556156 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-5fc0009c-284d-4541-891e-4514de1ed1eb None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:06:49.557366 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-5fc0009c-284d-4541-891e-4514de1ed1eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:49.585872 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:49.623362 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 656/1294] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:06:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:06:49.645729 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2340b656-a397-4713-9947-3a318f776a15 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:49.677207 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:49.748190 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2340b656-a397-4713-9947-3a318f776a15 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:49.785911 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d85f9e9-17d8-46b5-a250-c520d65a37b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:49.786450 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d85f9e9-17d8-46b5-a250-c520d65a37b6 None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:49.786733 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d85f9e9-17d8-46b5-a250-c520d65a37b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:49.787009 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d85f9e9-17d8-46b5-a250-c520d65a37b6 None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:49.787740 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d85f9e9-17d8-46b5-a250-c520d65a37b6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:49.789379 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d85f9e9-17d8-46b5-a250-c520d65a37b6 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:49.874468 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 639/1295] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Aug 4 21:06:49 2022] POST /identity/v3/services => generated 246 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:06:51.904683 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a919e30-24b6-4549-b5b6-ea851c78e7dd None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:51.905240 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a919e30-24b6-4549-b5b6-ea851c78e7dd None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:51.905240 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a919e30-24b6-4549-b5b6-ea851c78e7dd None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:51.906596 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 657/1296] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:06:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:06:51.912280 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba975a47-f94a-4a51-a68c-45e29493071a None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:51.912707 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba975a47-f94a-4a51-a68c-45e29493071a None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:51.912869 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba975a47-f94a-4a51-a68c-45e29493071a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:51.931469 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-ba975a47-f94a-4a51-a68c-45e29493071a None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:06:51.932754 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-ba975a47-f94a-4a51-a68c-45e29493071a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:52.003056 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 640/1297] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:06:51 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:06:52.025644 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a919e30-24b6-4549-b5b6-ea851c78e7dd None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:52.125810 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a919e30-24b6-4549-b5b6-ea851c78e7dd None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:52.158790 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b09486-c3eb-49cf-9704-3b36f7937ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:52.160068 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b09486-c3eb-49cf-9704-3b36f7937ebe None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:52.160347 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b09486-c3eb-49cf-9704-3b36f7937ebe None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:52.167830 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b09486-c3eb-49cf-9704-3b36f7937ebe None admin] PATH_INFO: `/v3/services/placement` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:52.167830 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b09486-c3eb-49cf-9704-3b36f7937ebe None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:52.167830 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b09486-c3eb-49cf-9704-3b36f7937ebe None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:52.171908 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-d6b09486-c3eb-49cf-9704-3b36f7937ebe None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 21:06:52.171908 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:06:52.171908 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 21:06:52.171908 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 21:06:52.171908 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 21:06:52.171908 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 21:06:52.171908 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 21:06:52.171908 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:06:52.171908 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 21:06:52.171908 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:06:52.171908 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 21:06:52.171908 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:06:52.171908 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:06:52.171908 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:06:52.171908 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:06:52.171908 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:06:52.171908 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:06:52.173044 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:06:52.173044 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:06:52.173044 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:06:52.173044 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:06:52.173044 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:06:52.173044 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:06:52.173044 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 21:06:52.173044 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 21:06:52.173044 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 21:06:52.173044 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 21:06:52.173044 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:06:52.173044 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:06:52.173044 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:06:52.173044 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:06:52.173044 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:06:52.173044 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:06:52.174208 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:06:52.174208 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:06:52.174208 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:06:52.174208 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:06:52.174208 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:06:52.174208 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:06:52.174208 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:06:52.174208 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:06:52.174208 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:06:52.174208 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:06:52.174208 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 21:06:52.174208 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:06:52.174208 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 21:06:52.174208 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:06:52.174208 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 658/1298] 10.222.0.25 () {56 vars in 1117 bytes} [Thu Aug 4 21:06:52 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:06:52.179370 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba975a47-f94a-4a51-a68c-45e29493071a None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:52.197743 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:52.278592 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba975a47-f94a-4a51-a68c-45e29493071a None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:52.312641 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c72556-6785-496d-a4bf-522088e5c5e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:52.313259 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c72556-6785-496d-a4bf-522088e5c5e8 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:52.313533 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c72556-6785-496d-a4bf-522088e5c5e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:52.313802 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c72556-6785-496d-a4bf-522088e5c5e8 None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:52.314462 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c72556-6785-496d-a4bf-522088e5c5e8 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:06:52.314762 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c72556-6785-496d-a4bf-522088e5c5e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:52.316212 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c72556-6785-496d-a4bf-522088e5c5e8 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:52.321154 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 641/1299] 10.222.0.25 () {56 vars in 1116 bytes} [Thu Aug 4 21:06:52 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:06:52.327102 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b09486-c3eb-49cf-9704-3b36f7937ebe None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:52.359922 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:52.437209 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b09486-c3eb-49cf-9704-3b36f7937ebe None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:52.475945 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdf8133-803b-4ecd-a4e4-25a1c4d5aa24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:52.476568 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf8133-803b-4ecd-a4e4-25a1c4d5aa24 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:52.476853 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf8133-803b-4ecd-a4e4-25a1c4d5aa24 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:52.477129 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf8133-803b-4ecd-a4e4-25a1c4d5aa24 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:52.477886 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdf8133-803b-4ecd-a4e4-25a1c4d5aa24 None admin] RBAC: Adding query filter params (interface=public, service_id=293b0a1a1e914f699a5e0ca4b53b4f3d, region_id=RegionOne) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:06:52.478195 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdf8133-803b-4ecd-a4e4-25a1c4d5aa24 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:52.479602 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdf8133-803b-4ecd-a4e4-25a1c4d5aa24 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:52.484546 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 659/1300] 10.222.0.25 () {56 vars in 1251 bytes} [Thu Aug 4 21:06:52 2022] GET /identity/v3/endpoints?service_id=293b0a1a1e914f699a5e0ca4b53b4f3d&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:06:52.490360 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c72556-6785-496d-a4bf-522088e5c5e8 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:52.572849 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:52.593620 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c72556-6785-496d-a4bf-522088e5c5e8 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:52.625927 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750b3b94-5edc-4d12-9a69-3a9ff2475cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:52.626555 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750b3b94-5edc-4d12-9a69-3a9ff2475cb1 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:52.626832 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750b3b94-5edc-4d12-9a69-3a9ff2475cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:52.627104 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750b3b94-5edc-4d12-9a69-3a9ff2475cb1 None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:52.628292 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750b3b94-5edc-4d12-9a69-3a9ff2475cb1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:52.629667 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750b3b94-5edc-4d12-9a69-3a9ff2475cb1 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:52.636456 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 642/1301] 10.222.0.25 () {56 vars in 1087 bytes} [Thu Aug 4 21:06:52 2022] GET /identity/v3/services => generated 2191 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:06:54.672445 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f006f7e9-e9fc-488d-9038-ab2f06e1190c None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:54.672960 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f006f7e9-e9fc-488d-9038-ab2f06e1190c None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:54.673055 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f006f7e9-e9fc-488d-9038-ab2f06e1190c None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:54.674336 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 660/1302] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:06:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:06:54.679880 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b93d4e-738e-4487-84de-72f4e6d78f7f None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:54.680297 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b93d4e-738e-4487-84de-72f4e6d78f7f None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:54.683290 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b93d4e-738e-4487-84de-72f4e6d78f7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:54.698931 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-e8b93d4e-738e-4487-84de-72f4e6d78f7f None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:06:54.700136 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-e8b93d4e-738e-4487-84de-72f4e6d78f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:54.765299 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 643/1303] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:06:54 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:06:54.788320 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f006f7e9-e9fc-488d-9038-ab2f06e1190c None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:54.906665 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:54.913667 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f006f7e9-e9fc-488d-9038-ab2f06e1190c None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:54.947819 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc0c8ba-e9d7-4cce-8f40-d7703f1b017f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:54.948381 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc0c8ba-e9d7-4cce-8f40-d7703f1b017f None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:54.948661 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc0c8ba-e9d7-4cce-8f40-d7703f1b017f None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:54.948929 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc0c8ba-e9d7-4cce-8f40-d7703f1b017f None admin] PATH_INFO: `/v3/services/placement` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:54.949546 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc0c8ba-e9d7-4cce-8f40-d7703f1b017f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:54.950906 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc0c8ba-e9d7-4cce-8f40-d7703f1b017f None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:54.960229 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-7dc0c8ba-e9d7-4cce-8f40-d7703f1b017f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 21:06:54.960229 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:06:54.960229 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 21:06:54.960229 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 21:06:54.960229 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 21:06:54.960229 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 21:06:54.960229 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 21:06:54.960229 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:06:54.960229 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 21:06:54.960229 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:06:54.960229 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 21:06:54.960229 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:06:54.960229 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:06:54.960229 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:06:54.960229 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:06:54.960229 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:06:54.960229 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:06:54.961383 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:06:54.961383 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:06:54.961383 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:06:54.961383 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:06:54.961383 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:06:54.961383 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:06:54.961383 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 21:06:54.961383 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 21:06:54.961383 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 21:06:54.961383 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 21:06:54.961383 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:06:54.961383 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:06:54.961383 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:06:54.961383 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:06:54.961383 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:06:54.961383 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:06:54.962256 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:06:54.962256 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Aug 04 21:06:54.962256 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:06:54.962256 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:06:54.962256 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:06:54.962256 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:06:54.962256 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:06:54.962256 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:06:54.962256 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:06:54.962256 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:06:54.962256 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 21:06:54.962256 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:06:54.962256 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 21:06:54.962256 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Aug 04 21:06:54.962256 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 661/1304] 10.222.0.25 () {56 vars in 1117 bytes} [Thu Aug 4 21:06:54 2022] GET /identity/v3/services/placement => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:06:54.967756 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b93d4e-738e-4487-84de-72f4e6d78f7f None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:55.043836 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:55.066254 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b93d4e-738e-4487-84de-72f4e6d78f7f None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:55.099955 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd221c3-493a-4bcb-9d15-d31fadd2cec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:55.100566 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd221c3-493a-4bcb-9d15-d31fadd2cec4 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:55.100845 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd221c3-493a-4bcb-9d15-d31fadd2cec4 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:55.101115 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd221c3-493a-4bcb-9d15-d31fadd2cec4 None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:55.101779 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd221c3-493a-4bcb-9d15-d31fadd2cec4 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:06:55.102077 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd221c3-493a-4bcb-9d15-d31fadd2cec4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:55.103476 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd221c3-493a-4bcb-9d15-d31fadd2cec4 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:55.108495 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 644/1305] 10.222.0.25 () {56 vars in 1116 bytes} [Thu Aug 4 21:06:54 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:06:55.114361 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc0c8ba-e9d7-4cce-8f40-d7703f1b017f None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:55.216735 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc0c8ba-e9d7-4cce-8f40-d7703f1b017f None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:55.250914 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efcc64d-63cc-4644-8b88-76baae010375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:55.251577 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efcc64d-63cc-4644-8b88-76baae010375 None admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:55.251866 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efcc64d-63cc-4644-8b88-76baae010375 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:55.254268 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efcc64d-63cc-4644-8b88-76baae010375 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:55.254268 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efcc64d-63cc-4644-8b88-76baae010375 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:55.254268 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efcc64d-63cc-4644-8b88-76baae010375 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:55.334886 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 662/1306] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:06:55 2022] POST /identity/v3/endpoints => generated 331 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:07:05.122348 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=30 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:07:05.329746 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:16:14.172165 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b9d34f-0a1a-4542-a039-5449057d0141 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:16:14.172752 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b9d34f-0a1a-4542-a039-5449057d0141 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:16:14.173042 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b9d34f-0a1a-4542-a039-5449057d0141 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:16:14.184056 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6664878f-8808-49ce-8a91-e5dd6a8ef1a3 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:16:14.196762 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6664878f-8808-49ce-8a91-e5dd6a8ef1a3 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:16:14.196762 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6664878f-8808-49ce-8a91-e5dd6a8ef1a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:16:14.197132 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 645/1307] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:16:14 2022] GET /identity => generated 270 bytes in 21 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:16:14.301537 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-6664878f-8808-49ce-8a91-e5dd6a8ef1a3 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:16:14.303173 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-6664878f-8808-49ce-8a91-e5dd6a8ef1a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:16:14.467854 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 663/1308] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:16:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:16:14.522851 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47afa0c-a2df-46ca-9fb4-39746f3d3fdb None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:16:14.524214 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47afa0c-a2df-46ca-9fb4-39746f3d3fdb None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:16:14.525902 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47afa0c-a2df-46ca-9fb4-39746f3d3fdb None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:16:14.535492 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 646/1309] 10.222.0.25 () {54 vars in 965 bytes} [Thu Aug 4 21:16:14 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:16:14.550513 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4bf139-e4ae-4a8f-9c1c-45947da44d51 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:16:14.550878 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4bf139-e4ae-4a8f-9c1c-45947da44d51 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:16:14.551159 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4bf139-e4ae-4a8f-9c1c-45947da44d51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:16:14.726167 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-2d4bf139-e4ae-4a8f-9c1c-45947da44d51 None None] MFA Rules not processed for user `08fcb91ee54344b5bb3870c585e25fdc`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:16:14.726167 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-2d4bf139-e4ae-4a8f-9c1c-45947da44d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:16:14.763033 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:16:14.853494 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 664/1310] 10.222.0.25 () {58 vars in 1062 bytes} [Thu Aug 4 21:16:14 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:16:14.865008 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47afa0c-a2df-46ca-9fb4-39746f3d3fdb None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:16:14.869433 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-d47afa0c-a2df-46ca-9fb4-39746f3d3fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:16:15.144925 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47afa0c-a2df-46ca-9fb4-39746f3d3fdb None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:16:15.144925 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cad763-d310-4e45-88fa-8276f52c2ff9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:16:15.144925 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cad763-d310-4e45-88fa-8276f52c2ff9 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:16:15.144925 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cad763-d310-4e45-88fa-8276f52c2ff9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:16:15.145316 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cad763-d310-4e45-88fa-8276f52c2ff9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:16:15.145316 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-b7cad763-d310-4e45-88fa-8276f52c2ff9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:16:15.195851 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:16:15.264671 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cad763-d310-4e45-88fa-8276f52c2ff9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:16:15.266173 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cad763-d310-4e45-88fa-8276f52c2ff9 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:16:15.288902 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 647/1311] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:16:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:16:18.178475 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993be997-8ef7-4790-bd00-3c041ca3db76 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:16:18.179030 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993be997-8ef7-4790-bd00-3c041ca3db76 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:16:18.179112 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993be997-8ef7-4790-bd00-3c041ca3db76 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:16:18.180549 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 665/1312] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:16:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:16:18.186056 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3dc303-9ec6-4095-8c8c-bb27a5332bf2 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:16:18.186539 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3dc303-9ec6-4095-8c8c-bb27a5332bf2 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:16:18.186698 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3dc303-9ec6-4095-8c8c-bb27a5332bf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:16:18.274190 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-4d3dc303-9ec6-4095-8c8c-bb27a5332bf2 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:16:18.275445 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-4d3dc303-9ec6-4095-8c8c-bb27a5332bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:16:18.401187 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 648/1313] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:16:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:16:18.454418 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4817d6-671d-498e-8ce3-f00497f93844 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:16:18.454823 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4817d6-671d-498e-8ce3-f00497f93844 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:16:18.455184 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4817d6-671d-498e-8ce3-f00497f93844 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:16:18.457146 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 666/1314] 10.222.0.25 () {54 vars in 965 bytes} [Thu Aug 4 21:16:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:16:18.464651 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8de1c0-97ac-4098-9eca-792f41c93d15 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:16:18.465048 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8de1c0-97ac-4098-9eca-792f41c93d15 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:16:18.465241 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8de1c0-97ac-4098-9eca-792f41c93d15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:16:18.478687 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:16:18.559902 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-4b8de1c0-97ac-4098-9eca-792f41c93d15 None None] MFA Rules not processed for user `08fcb91ee54344b5bb3870c585e25fdc`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:16:18.561164 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-4b8de1c0-97ac-4098-9eca-792f41c93d15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:16:18.639470 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 649/1315] 10.222.0.25 () {58 vars in 1062 bytes} [Thu Aug 4 21:16:18 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:16:18.648576 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4817d6-671d-498e-8ce3-f00497f93844 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:16:18.650159 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-7f4817d6-671d-498e-8ce3-f00497f93844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:16:18.786743 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4817d6-671d-498e-8ce3-f00497f93844 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:16:18.868625 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:16:18.870987 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a80406d-037b-4f95-8914-9ecac35b7fa5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:16:18.871645 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a80406d-037b-4f95-8914-9ecac35b7fa5 service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:16:18.871932 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a80406d-037b-4f95-8914-9ecac35b7fa5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:16:18.872207 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a80406d-037b-4f95-8914-9ecac35b7fa5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:16:18.873886 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-9a80406d-037b-4f95-8914-9ecac35b7fa5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:16:18.990885 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a80406d-037b-4f95-8914-9ecac35b7fa5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:16:18.992378 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a80406d-037b-4f95-8914-9ecac35b7fa5 service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:16:19.012457 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 667/1316] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:16:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:16:21.971931 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c43956-706f-4134-9fe1-fc3d1bd9faeb None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:16:21.972502 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c43956-706f-4134-9fe1-fc3d1bd9faeb None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:16:21.972502 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c43956-706f-4134-9fe1-fc3d1bd9faeb None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:16:21.973872 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 650/1317] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:16:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:16:21.980487 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a1fc3e-b1fc-4022-afc5-aef988e7633a None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:16:21.980487 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a1fc3e-b1fc-4022-afc5-aef988e7633a None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:16:21.981096 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a1fc3e-b1fc-4022-afc5-aef988e7633a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:16:22.050216 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-d1a1fc3e-b1fc-4022-afc5-aef988e7633a None None] MFA Rules not processed for user `d562ab3d5d014f1dbf980dedad3e33c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:16:22.051576 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-d1a1fc3e-b1fc-4022-afc5-aef988e7633a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:16:22.108679 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:16:22.158505 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 668/1318] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:16:21 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:16:22.205210 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c43956-706f-4134-9fe1-fc3d1bd9faeb None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:16:22.207278 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-b6c43956-706f-4134-9fe1-fc3d1bd9faeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:16:22.329942 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c43956-706f-4134-9fe1-fc3d1bd9faeb None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:16:22.333094 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:16:22.409555 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b078c52-72e1-459b-9ac4-ad72a085647d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:16:22.409840 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b078c52-72e1-459b-9ac4-ad72a085647d service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:16:22.410171 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b078c52-72e1-459b-9ac4-ad72a085647d service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:16:22.410454 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b078c52-72e1-459b-9ac4-ad72a085647d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:16:22.412162 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-9b078c52-72e1-459b-9ac4-ad72a085647d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:16:22.550054 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b078c52-72e1-459b-9ac4-ad72a085647d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:16:22.552046 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b078c52-72e1-459b-9ac4-ad72a085647d service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:16:22.575910 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 651/1319] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:16:22 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:16:26.090118 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdebfdf-3cc0-4c60-8e0e-92e12ae4bbeb None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:16:26.090898 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdebfdf-3cc0-4c60-8e0e-92e12ae4bbeb None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:16:26.091050 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdebfdf-3cc0-4c60-8e0e-92e12ae4bbeb None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:16:26.093025 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 669/1320] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:16:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:16:26.099537 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b022e-e68f-4298-98dc-6d258ae85324 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:16:26.100311 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b022e-e68f-4298-98dc-6d258ae85324 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:16:26.100311 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b022e-e68f-4298-98dc-6d258ae85324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:16:26.168457 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:16:26.209272 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-5c3b022e-e68f-4298-98dc-6d258ae85324 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:16:26.210709 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-5c3b022e-e68f-4298-98dc-6d258ae85324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:16:26.336086 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 652/1321] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:16:26 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:16:26.376292 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdebfdf-3cc0-4c60-8e0e-92e12ae4bbeb None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:16:26.377872 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-ebdebfdf-3cc0-4c60-8e0e-92e12ae4bbeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:16:26.511916 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdebfdf-3cc0-4c60-8e0e-92e12ae4bbeb None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:16:26.597032 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a011abe3-9a81-4ae3-a9a3-0c371ac554df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:16:26.597661 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a011abe3-9a81-4ae3-a9a3-0c371ac554df service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:16:26.597949 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a011abe3-9a81-4ae3-a9a3-0c371ac554df service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:16:26.598218 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a011abe3-9a81-4ae3-a9a3-0c371ac554df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:16:26.599929 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-a011abe3-9a81-4ae3-a9a3-0c371ac554df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:16:26.608078 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:16:26.733606 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a011abe3-9a81-4ae3-a9a3-0c371ac554df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:16:26.735068 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a011abe3-9a81-4ae3-a9a3-0c371ac554df service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:16:26.756066 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 670/1322] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:16:26 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:16:30.974537 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ee0b6e-3e98-403f-a8a0-cae205661203 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:16:30.975494 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ee0b6e-3e98-403f-a8a0-cae205661203 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:16:30.975494 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ee0b6e-3e98-403f-a8a0-cae205661203 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:16:30.976722 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 653/1323] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:16:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:16:30.983623 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e901a2-d676-4e1e-8a92-a8a027933a65 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:16:30.984119 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e901a2-d676-4e1e-8a92-a8a027933a65 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:16:30.984210 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e901a2-d676-4e1e-8a92-a8a027933a65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:16:31.066669 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-98e901a2-d676-4e1e-8a92-a8a027933a65 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:16:31.068854 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-98e901a2-d676-4e1e-8a92-a8a027933a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:16:31.136910 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:16:31.184172 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 671/1324] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:16:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:16:31.230607 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ee0b6e-3e98-403f-a8a0-cae205661203 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:16:31.232210 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-06ee0b6e-3e98-403f-a8a0-cae205661203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:16:31.380188 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ee0b6e-3e98-403f-a8a0-cae205661203 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:16:31.395678 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:16:31.493189 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0899cecd-51eb-47cc-8f8c-b992527ab04c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:16:31.493723 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0899cecd-51eb-47cc-8f8c-b992527ab04c service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:16:31.494003 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0899cecd-51eb-47cc-8f8c-b992527ab04c service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:16:31.494276 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0899cecd-51eb-47cc-8f8c-b992527ab04c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:16:31.496850 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-0899cecd-51eb-47cc-8f8c-b992527ab04c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:16:31.622570 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0899cecd-51eb-47cc-8f8c-b992527ab04c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:16:31.624084 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0899cecd-51eb-47cc-8f8c-b992527ab04c service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:16:31.643474 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 654/1325] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:16:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:16:33.791537 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a4aaf0-daf6-439d-97dc-5a18bfaef863 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:16:33.792708 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a4aaf0-daf6-439d-97dc-5a18bfaef863 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:16:33.792708 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a4aaf0-daf6-439d-97dc-5a18bfaef863 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:16:33.794125 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 672/1326] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:16:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:16:33.799453 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c48017-1663-4c73-a3c6-6ff78a98daef None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:16:33.799925 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c48017-1663-4c73-a3c6-6ff78a98daef None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:16:33.800019 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c48017-1663-4c73-a3c6-6ff78a98daef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:16:33.859945 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:16:34.006182 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-a8c48017-1663-4c73-a3c6-6ff78a98daef None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:16:34.007566 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-a8c48017-1663-4c73-a3c6-6ff78a98daef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:16:34.112607 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 655/1327] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:16:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:16:34.148573 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a4aaf0-daf6-439d-97dc-5a18bfaef863 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:16:34.150122 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-f3a4aaf0-daf6-439d-97dc-5a18bfaef863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:16:34.271948 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a4aaf0-daf6-439d-97dc-5a18bfaef863 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:16:34.482606 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c90447-31a8-4db6-922b-0a7b582bdd48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:16:34.483282 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c90447-31a8-4db6-922b-0a7b582bdd48 service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:16:34.483626 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c90447-31a8-4db6-922b-0a7b582bdd48 service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:16:34.483900 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c90447-31a8-4db6-922b-0a7b582bdd48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:16:34.485659 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-39c90447-31a8-4db6-922b-0a7b582bdd48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:16:34.505681 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:16:34.605034 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c90447-31a8-4db6-922b-0a7b582bdd48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:16:34.606506 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c90447-31a8-4db6-922b-0a7b582bdd48 service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:16:34.626057 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 673/1328] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:16:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:16:36.738286 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55951040-388e-4fcf-a76a-66e79d445217 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:16:36.739111 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55951040-388e-4fcf-a76a-66e79d445217 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:16:36.739111 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55951040-388e-4fcf-a76a-66e79d445217 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:16:36.740524 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 656/1329] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:16:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 21:16:36.747787 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4e8466-4cce-496a-a0e8-350fd4c65c2d None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:16:36.747787 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4e8466-4cce-496a-a0e8-350fd4c65c2d None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:16:36.748557 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4e8466-4cce-496a-a0e8-350fd4c65c2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:16:36.822417 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-ec4e8466-4cce-496a-a0e8-350fd4c65c2d None None] MFA Rules not processed for user `d562ab3d5d014f1dbf980dedad3e33c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:16:36.823671 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-ec4e8466-4cce-496a-a0e8-350fd4c65c2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:16:36.894019 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:16:36.935056 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 674/1330] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:16:36 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:16:36.974959 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55951040-388e-4fcf-a76a-66e79d445217 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:16:36.976880 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-55951040-388e-4fcf-a76a-66e79d445217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:16:37.123334 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55951040-388e-4fcf-a76a-66e79d445217 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:16:37.149780 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:16:37.268048 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07dfa46-2864-401a-b60c-21fe8ee6cbd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:16:37.268713 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07dfa46-2864-401a-b60c-21fe8ee6cbd9 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:16:37.269158 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07dfa46-2864-401a-b60c-21fe8ee6cbd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:16:37.269436 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07dfa46-2864-401a-b60c-21fe8ee6cbd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:16:37.271234 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-c07dfa46-2864-401a-b60c-21fe8ee6cbd9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:16:37.421750 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07dfa46-2864-401a-b60c-21fe8ee6cbd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:16:37.423214 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07dfa46-2864-401a-b60c-21fe8ee6cbd9 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:16:37.442540 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 657/1331] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:16:36 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:16:40.111795 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399ec83a-4524-4538-8284-21444e3033ae None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:16:40.112613 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399ec83a-4524-4538-8284-21444e3033ae None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:16:40.112613 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399ec83a-4524-4538-8284-21444e3033ae None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:16:40.113638 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 675/1332] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:16:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 21:16:40.119590 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da59e0db-3d62-4120-915c-ea8e6def4bab None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:16:40.121188 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da59e0db-3d62-4120-915c-ea8e6def4bab None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:16:40.121188 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da59e0db-3d62-4120-915c-ea8e6def4bab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:16:40.216271 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:16:40.220756 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-da59e0db-3d62-4120-915c-ea8e6def4bab None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:16:40.222037 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-da59e0db-3d62-4120-915c-ea8e6def4bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:16:40.328380 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 658/1333] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:16:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:16:40.369885 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399ec83a-4524-4538-8284-21444e3033ae None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:16:40.371685 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-399ec83a-4524-4538-8284-21444e3033ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:16:40.490142 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399ec83a-4524-4538-8284-21444e3033ae None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:16:40.566801 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853b4086-bbe1-417e-a44c-7e86945f08b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:16:40.567537 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853b4086-bbe1-417e-a44c-7e86945f08b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:16:40.567818 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853b4086-bbe1-417e-a44c-7e86945f08b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:16:40.568088 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853b4086-bbe1-417e-a44c-7e86945f08b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:16:40.569706 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-853b4086-bbe1-417e-a44c-7e86945f08b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:16:40.598344 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:16:40.698854 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853b4086-bbe1-417e-a44c-7e86945f08b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:16:40.700334 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853b4086-bbe1-417e-a44c-7e86945f08b9 service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:16:40.720901 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 676/1334] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:16:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:16:42.836728 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66421c0-f015-40a0-83d3-da36fcc7f565 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:16:42.837322 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66421c0-f015-40a0-83d3-da36fcc7f565 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:16:42.837322 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66421c0-f015-40a0-83d3-da36fcc7f565 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:16:42.841716 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 659/1335] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:16:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:16:42.845446 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71072119-e1f6-4042-abc4-c1e7bdacbade None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:16:42.845955 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71072119-e1f6-4042-abc4-c1e7bdacbade None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:16:42.846401 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71072119-e1f6-4042-abc4-c1e7bdacbade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:16:42.927186 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-71072119-e1f6-4042-abc4-c1e7bdacbade None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:16:42.928542 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-71072119-e1f6-4042-abc4-c1e7bdacbade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:16:43.008647 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:16:43.031658 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 677/1336] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:16:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:16:43.074292 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66421c0-f015-40a0-83d3-da36fcc7f565 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:16:43.076593 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-d66421c0-f015-40a0-83d3-da36fcc7f565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:16:43.185116 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66421c0-f015-40a0-83d3-da36fcc7f565 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:16:43.214194 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:16:43.266237 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e612ce-fe1b-46e2-a023-229bdd2f699e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:16:43.266872 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e612ce-fe1b-46e2-a023-229bdd2f699e service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:16:43.267158 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e612ce-fe1b-46e2-a023-229bdd2f699e service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:16:43.267469 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e612ce-fe1b-46e2-a023-229bdd2f699e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:16:43.269129 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-95e612ce-fe1b-46e2-a023-229bdd2f699e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:16:43.389770 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e612ce-fe1b-46e2-a023-229bdd2f699e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:16:43.391231 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e612ce-fe1b-46e2-a023-229bdd2f699e service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:16:43.411540 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 660/1337] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:16:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:16:46.301937 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85606baf-5e02-4b05-b0ca-646c7144e6f4 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:16:46.302572 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85606baf-5e02-4b05-b0ca-646c7144e6f4 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:16:46.302572 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85606baf-5e02-4b05-b0ca-646c7144e6f4 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:16:46.304140 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 678/1338] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:16:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:16:46.309919 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ddc66-64c6-48c1-9d38-314d65ae2867 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:16:46.310390 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ddc66-64c6-48c1-9d38-314d65ae2867 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:16:46.310493 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ddc66-64c6-48c1-9d38-314d65ae2867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:16:46.383008 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-770ddc66-64c6-48c1-9d38-314d65ae2867 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:16:46.384413 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-770ddc66-64c6-48c1-9d38-314d65ae2867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:16:46.392775 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:16:46.502018 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 661/1339] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:16:46 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:16:46.545808 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85606baf-5e02-4b05-b0ca-646c7144e6f4 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:16:46.547341 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-85606baf-5e02-4b05-b0ca-646c7144e6f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:16:46.655431 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85606baf-5e02-4b05-b0ca-646c7144e6f4 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:16:46.772566 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2653f432-262b-4716-8bf2-900529100a80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:16:46.773180 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2653f432-262b-4716-8bf2-900529100a80 service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:16:46.775307 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2653f432-262b-4716-8bf2-900529100a80 service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:16:46.775307 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2653f432-262b-4716-8bf2-900529100a80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:16:46.775542 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-2653f432-262b-4716-8bf2-900529100a80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:16:46.825980 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:16:46.920449 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2653f432-262b-4716-8bf2-900529100a80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:16:46.921910 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2653f432-262b-4716-8bf2-900529100a80 service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:16:46.942685 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 679/1340] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:16:46 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:16:51.829018 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f204e-5ae1-4146-9668-167987926159 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:16:51.829566 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f204e-5ae1-4146-9668-167987926159 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:16:51.829566 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f204e-5ae1-4146-9668-167987926159 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:16:51.831054 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 662/1341] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:16:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:16:51.836661 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c2dfb-3468-48f6-a784-35a57b095f47 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:16:51.837119 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c2dfb-3468-48f6-a784-35a57b095f47 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:16:51.837767 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c2dfb-3468-48f6-a784-35a57b095f47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:16:51.908410 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-867c2dfb-3468-48f6-a784-35a57b095f47 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:16:51.909639 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-867c2dfb-3468-48f6-a784-35a57b095f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:16:52.000473 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:16:52.013001 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 680/1342] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:16:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:16:52.052642 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296f204e-5ae1-4146-9668-167987926159 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:16:52.054473 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-296f204e-5ae1-4146-9668-167987926159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:16:52.167612 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296f204e-5ae1-4146-9668-167987926159 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:16:52.208148 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:16:52.251766 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da88a0d3-8f87-4035-bed3-3c25d57bdd04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:16:52.252464 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da88a0d3-8f87-4035-bed3-3c25d57bdd04 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:16:52.252752 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da88a0d3-8f87-4035-bed3-3c25d57bdd04 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:16:52.253026 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da88a0d3-8f87-4035-bed3-3c25d57bdd04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:16:52.254657 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-da88a0d3-8f87-4035-bed3-3c25d57bdd04 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:16:52.374628 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da88a0d3-8f87-4035-bed3-3c25d57bdd04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:16:52.375880 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da88a0d3-8f87-4035-bed3-3c25d57bdd04 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:16:52.394482 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 663/1343] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:16:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:16:55.727922 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2851084-0859-4ad3-b574-d519f65d2b81 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:16:55.728460 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2851084-0859-4ad3-b574-d519f65d2b81 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:16:55.728587 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2851084-0859-4ad3-b574-d519f65d2b81 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:16:55.729993 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 681/1344] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:16:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:16:55.735360 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70c114b-a6c8-4a67-9af3-81c4a8bd6752 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:16:55.735842 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70c114b-a6c8-4a67-9af3-81c4a8bd6752 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:16:55.736053 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70c114b-a6c8-4a67-9af3-81c4a8bd6752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:16:55.806682 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-c70c114b-a6c8-4a67-9af3-81c4a8bd6752 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:16:55.808629 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-c70c114b-a6c8-4a67-9af3-81c4a8bd6752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:16:55.828198 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:16:55.923545 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 664/1345] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:16:55 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:16:55.962019 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2851084-0859-4ad3-b574-d519f65d2b81 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:16:55.963673 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-e2851084-0859-4ad3-b574-d519f65d2b81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:16:56.074198 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2851084-0859-4ad3-b574-d519f65d2b81 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:16:56.163951 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fde93a-83f1-4cb6-91f9-4e8fb61696f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:16:56.164583 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fde93a-83f1-4cb6-91f9-4e8fb61696f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:16:56.164867 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fde93a-83f1-4cb6-91f9-4e8fb61696f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:16:56.165141 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fde93a-83f1-4cb6-91f9-4e8fb61696f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:16:56.166890 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-d5fde93a-83f1-4cb6-91f9-4e8fb61696f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:16:56.215065 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:16:56.287000 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fde93a-83f1-4cb6-91f9-4e8fb61696f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:16:56.288801 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fde93a-83f1-4cb6-91f9-4e8fb61696f8 service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:16:56.316925 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 682/1346] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:16:55 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:17:01.166549 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3759dd97-b21b-462b-b3df-beb8f44299ac None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:17:01.167421 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3759dd97-b21b-462b-b3df-beb8f44299ac None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:17:01.167421 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3759dd97-b21b-462b-b3df-beb8f44299ac None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:17:01.168907 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 665/1347] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:17:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:17:01.174516 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21719b58-eb0a-4f5b-8377-836eab542244 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:17:01.175054 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21719b58-eb0a-4f5b-8377-836eab542244 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:17:01.175054 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21719b58-eb0a-4f5b-8377-836eab542244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:17:01.264365 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-21719b58-eb0a-4f5b-8377-836eab542244 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:17:01.265602 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-21719b58-eb0a-4f5b-8377-836eab542244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:17:01.374642 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:17:01.377416 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 683/1348] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:17:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:17:01.418080 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3759dd97-b21b-462b-b3df-beb8f44299ac None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:17:01.419831 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-3759dd97-b21b-462b-b3df-beb8f44299ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:17:01.550888 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3759dd97-b21b-462b-b3df-beb8f44299ac None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:17:01.605067 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:17:01.636547 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b94cc20-043a-405b-87cc-00e13735dc64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:17:01.637170 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b94cc20-043a-405b-87cc-00e13735dc64 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:17:01.637445 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b94cc20-043a-405b-87cc-00e13735dc64 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:17:01.637713 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b94cc20-043a-405b-87cc-00e13735dc64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:17:01.639355 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-9b94cc20-043a-405b-87cc-00e13735dc64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:17:01.755345 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b94cc20-043a-405b-87cc-00e13735dc64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:17:01.757147 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b94cc20-043a-405b-87cc-00e13735dc64 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:17:01.776741 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 666/1349] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:17:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:17:07.310661 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedfd9e7-c3c1-40be-9cda-f1e137d95dc0 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:17:07.311199 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedfd9e7-c3c1-40be-9cda-f1e137d95dc0 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:17:07.311199 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedfd9e7-c3c1-40be-9cda-f1e137d95dc0 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:17:07.313357 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 684/1350] 10.222.0.25 () {54 vars in 921 bytes} [Thu Aug 4 21:17:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:17:07.320058 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48205dee-2d63-43eb-87dc-f2ffb6963cda None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:17:07.320561 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48205dee-2d63-43eb-87dc-f2ffb6963cda None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:17:07.320914 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48205dee-2d63-43eb-87dc-f2ffb6963cda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:17:07.452895 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-48205dee-2d63-43eb-87dc-f2ffb6963cda None None] MFA Rules not processed for user `67d44a0098ab4c36b14b36e9e59ae75c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:17:07.454415 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-48205dee-2d63-43eb-87dc-f2ffb6963cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:17:07.483775 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:17:07.650609 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 667/1351] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Aug 4 21:17:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:17:13.038515 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b4d26b-6de7-45d2-b901-356052b6e984 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:17:13.039047 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b4d26b-6de7-45d2-b901-356052b6e984 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:17:13.040681 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b4d26b-6de7-45d2-b901-356052b6e984 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:17:13.043180 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 685/1352] 10.222.0.25 () {54 vars in 921 bytes} [Thu Aug 4 21:17:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:17:13.051742 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aedd382-09a8-4c69-bb03-7fa8e1361d50 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:17:13.052265 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aedd382-09a8-4c69-bb03-7fa8e1361d50 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:17:13.052265 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aedd382-09a8-4c69-bb03-7fa8e1361d50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:17:13.169890 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-0aedd382-09a8-4c69-bb03-7fa8e1361d50 None None] MFA Rules not processed for user `67d44a0098ab4c36b14b36e9e59ae75c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:17:13.171123 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-0aedd382-09a8-4c69-bb03-7fa8e1361d50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:17:13.277275 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:17:13.338171 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 668/1353] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Aug 4 21:17:13 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:17:15.215925 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b524d9-f661-4d86-ad19-876342701081 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:17:15.216649 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b524d9-f661-4d86-ad19-876342701081 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:17:15.216649 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b524d9-f661-4d86-ad19-876342701081 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:17:15.217814 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 686/1354] 10.222.0.25 () {54 vars in 921 bytes} [Thu Aug 4 21:17:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:17:15.223903 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e54f999-95a1-4f37-af0f-f0dd69a6bf3a None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:17:15.224425 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e54f999-95a1-4f37-af0f-f0dd69a6bf3a None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:17:15.224425 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e54f999-95a1-4f37-af0f-f0dd69a6bf3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:17:15.354969 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-9e54f999-95a1-4f37-af0f-f0dd69a6bf3a None None] MFA Rules not processed for user `67d44a0098ab4c36b14b36e9e59ae75c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:17:15.356241 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-9e54f999-95a1-4f37-af0f-f0dd69a6bf3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:17:15.565614 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 669/1355] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Aug 4 21:17:15 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:17:19.590105 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca6b0e5-6a05-4f73-ae8c-417145d58d48 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:17:19.590967 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca6b0e5-6a05-4f73-ae8c-417145d58d48 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:17:19.590967 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca6b0e5-6a05-4f73-ae8c-417145d58d48 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:17:19.593079 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 687/1356] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:17:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:17:19.602017 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904c9d6d-0c5a-4019-867a-efff1e06b459 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:17:19.602776 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904c9d6d-0c5a-4019-867a-efff1e06b459 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:17:19.602776 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904c9d6d-0c5a-4019-867a-efff1e06b459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:17:19.707772 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:17:19.786914 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-904c9d6d-0c5a-4019-867a-efff1e06b459 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:17:19.789301 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-904c9d6d-0c5a-4019-867a-efff1e06b459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:17:20.030388 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 670/1357] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:17:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:17:20.123178 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1e527f-41c0-4cdd-8700-4c362a1823f1 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:17:20.123842 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1e527f-41c0-4cdd-8700-4c362a1823f1 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:17:20.124628 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1e527f-41c0-4cdd-8700-4c362a1823f1 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:17:20.128553 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 688/1358] 10.222.0.25 () {54 vars in 962 bytes} [Thu Aug 4 21:17:20 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 21:17:20.149520 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b477a0c-5197-452c-a6bb-ce6ff2fdbfbb None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:17:20.150599 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b477a0c-5197-452c-a6bb-ce6ff2fdbfbb None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:17:20.151275 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b477a0c-5197-452c-a6bb-ce6ff2fdbfbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:17:20.340228 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-7b477a0c-5197-452c-a6bb-ce6ff2fdbfbb None None] MFA Rules not processed for user `6a2ce1df87fa47e5be301916bf3245fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:17:20.342664 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-7b477a0c-5197-452c-a6bb-ce6ff2fdbfbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:17:20.462756 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:17:20.623152 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 671/1359] 10.222.0.25 () {58 vars in 1059 bytes} [Thu Aug 4 21:17:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 480 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:17:20.648827 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1e527f-41c0-4cdd-8700-4c362a1823f1 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:17:20.649550 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-ad1e527f-41c0-4cdd-8700-4c362a1823f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:17:20.946498 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1e527f-41c0-4cdd-8700-4c362a1823f1 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:17:21.095771 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:17:21.167005 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da08b08-524b-4c4d-b0e1-16667a5bcbb8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a2ce1df87fa47e5be301916bf3245fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:17:21.167005 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da08b08-524b-4c4d-b0e1-16667a5bcbb8 service nova] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:17:21.167005 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da08b08-524b-4c4d-b0e1-16667a5bcbb8 service nova] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:17:21.167005 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da08b08-524b-4c4d-b0e1-16667a5bcbb8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:17:21.172100 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-6da08b08-524b-4c4d-b0e1-16667a5bcbb8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:17:21.388482 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da08b08-524b-4c4d-b0e1-16667a5bcbb8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:17:21.391083 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da08b08-524b-4c4d-b0e1-16667a5bcbb8 service nova] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:17:21.425914 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 689/1360] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:17:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 779 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:17:24.491342 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eef6dc-36ac-405d-bd32-38b7647460ac None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:17:24.492046 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eef6dc-36ac-405d-bd32-38b7647460ac None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:17:24.492205 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eef6dc-36ac-405d-bd32-38b7647460ac None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:17:24.495288 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 672/1361] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:17:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:17:24.501895 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444d18d3-88d0-4e5f-b1d0-452c1277f3e9 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:17:24.502691 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444d18d3-88d0-4e5f-b1d0-452c1277f3e9 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:17:24.502868 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444d18d3-88d0-4e5f-b1d0-452c1277f3e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:17:24.625515 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-444d18d3-88d0-4e5f-b1d0-452c1277f3e9 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:17:24.627475 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-444d18d3-88d0-4e5f-b1d0-452c1277f3e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:17:24.701977 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:17:24.815474 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 690/1362] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:17:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:17:24.874895 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eef6dc-36ac-405d-bd32-38b7647460ac None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:17:24.878086 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-f2eef6dc-36ac-405d-bd32-38b7647460ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:17:25.242686 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eef6dc-36ac-405d-bd32-38b7647460ac None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:17:25.270474 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:17:25.489715 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642038aa-9fdc-4d8d-b63b-6cf2204ba114 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a2ce1df87fa47e5be301916bf3245fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:17:25.490749 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642038aa-9fdc-4d8d-b63b-6cf2204ba114 service nova] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:17:25.491240 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642038aa-9fdc-4d8d-b63b-6cf2204ba114 service nova] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:17:25.491774 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642038aa-9fdc-4d8d-b63b-6cf2204ba114 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:17:25.495358 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-642038aa-9fdc-4d8d-b63b-6cf2204ba114 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:17:25.625749 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642038aa-9fdc-4d8d-b63b-6cf2204ba114 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:17:25.627949 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642038aa-9fdc-4d8d-b63b-6cf2204ba114 service nova] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:17:25.655934 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 673/1363] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:17:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 782 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:17:29.036828 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27647fc6-e9cf-4e7e-a1e4-81b122c8ede5 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:17:29.037387 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27647fc6-e9cf-4e7e-a1e4-81b122c8ede5 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:17:29.037492 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27647fc6-e9cf-4e7e-a1e4-81b122c8ede5 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:17:29.039507 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 691/1364] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:17:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:17:29.047219 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a30a0d-7b5f-4c9d-9f13-844eb3328e32 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:17:29.047811 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a30a0d-7b5f-4c9d-9f13-844eb3328e32 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:17:29.047811 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a30a0d-7b5f-4c9d-9f13-844eb3328e32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:17:29.074616 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:17:29.164791 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-d3a30a0d-7b5f-4c9d-9f13-844eb3328e32 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:17:29.166805 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-d3a30a0d-7b5f-4c9d-9f13-844eb3328e32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:17:29.340161 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 674/1365] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:17:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:17:29.414870 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b395f71-3573-4e7a-a856-e8db6ffa23f9 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:17:29.415436 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b395f71-3573-4e7a-a856-e8db6ffa23f9 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:17:29.415819 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b395f71-3573-4e7a-a856-e8db6ffa23f9 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:17:29.417754 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 692/1366] 10.222.0.25 () {54 vars in 962 bytes} [Thu Aug 4 21:17:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:17:29.430295 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e605ced5-7924-4a37-9b98-acc8af3963bf None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:17:29.433851 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e605ced5-7924-4a37-9b98-acc8af3963bf None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:17:29.434187 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e605ced5-7924-4a37-9b98-acc8af3963bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:17:29.614434 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-e605ced5-7924-4a37-9b98-acc8af3963bf None None] MFA Rules not processed for user `6a2ce1df87fa47e5be301916bf3245fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:17:29.616311 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-e605ced5-7924-4a37-9b98-acc8af3963bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:17:29.695065 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:17:29.859356 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 675/1367] 10.222.0.25 () {58 vars in 1059 bytes} [Thu Aug 4 21:17:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 431 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:17:29.871852 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b395f71-3573-4e7a-a856-e8db6ffa23f9 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:17:29.874956 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-8b395f71-3573-4e7a-a856-e8db6ffa23f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:17:30.162131 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b395f71-3573-4e7a-a856-e8db6ffa23f9 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:17:30.191556 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:17:30.333886 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42eb9afa-7867-4da9-8540-eb8cf2e97c7a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a2ce1df87fa47e5be301916bf3245fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:17:30.334490 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eb9afa-7867-4da9-8540-eb8cf2e97c7a service nova] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:17:30.334840 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eb9afa-7867-4da9-8540-eb8cf2e97c7a service nova] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:17:30.335122 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eb9afa-7867-4da9-8540-eb8cf2e97c7a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:17:30.337003 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-42eb9afa-7867-4da9-8540-eb8cf2e97c7a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:17:30.482321 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42eb9afa-7867-4da9-8540-eb8cf2e97c7a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:17:30.485002 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42eb9afa-7867-4da9-8540-eb8cf2e97c7a service nova] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:17:30.520295 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 693/1368] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:17:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 650 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:17:31.182510 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feba6b4-3cfb-4813-b73d-161dc403f7ab None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:17:31.183098 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feba6b4-3cfb-4813-b73d-161dc403f7ab None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:17:31.183098 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feba6b4-3cfb-4813-b73d-161dc403f7ab None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:17:31.187723 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 676/1369] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:17:31 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:17:31.194444 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dff22d-574b-4dcf-90f8-8ac39f9d8a5d None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:17:31.195020 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dff22d-574b-4dcf-90f8-8ac39f9d8a5d None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:17:31.195184 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dff22d-574b-4dcf-90f8-8ac39f9d8a5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:17:31.264283 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:17:31.349977 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-50dff22d-574b-4dcf-90f8-8ac39f9d8a5d None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:17:31.352177 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-50dff22d-574b-4dcf-90f8-8ac39f9d8a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:17:31.557633 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 694/1370] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:17:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:17:32.075383 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea01bb3-5d02-4f7c-acfd-ef38b1a29878 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:17:32.075383 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea01bb3-5d02-4f7c-acfd-ef38b1a29878 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:17:32.076175 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea01bb3-5d02-4f7c-acfd-ef38b1a29878 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:17:32.078412 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 677/1371] 10.222.0.25 () {54 vars in 964 bytes} [Thu Aug 4 21:17:32 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 21:17:32.086181 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541167ac-6bc4-4d6b-ae5c-4fadde2f1e87 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:17:32.086681 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541167ac-6bc4-4d6b-ae5c-4fadde2f1e87 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:17:32.087066 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541167ac-6bc4-4d6b-ae5c-4fadde2f1e87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:17:32.283666 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-541167ac-6bc4-4d6b-ae5c-4fadde2f1e87 None None] MFA Rules not processed for user `e31f2728056b460fa474478eb85ca1d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:17:32.285717 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-541167ac-6bc4-4d6b-ae5c-4fadde2f1e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:17:32.374822 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:17:32.429646 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 695/1372] 10.222.0.25 () {58 vars in 1061 bytes} [Thu Aug 4 21:17:32 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:17:32.439796 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea01bb3-5d02-4f7c-acfd-ef38b1a29878 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:17:32.443413 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-2ea01bb3-5d02-4f7c-acfd-ef38b1a29878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:17:32.620410 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea01bb3-5d02-4f7c-acfd-ef38b1a29878 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:17:32.684864 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:17:32.738556 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0d8899-686d-49cf-98a8-2e2a7d68dc45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e31f2728056b460fa474478eb85ca1d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:17:32.738920 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0d8899-686d-49cf-98a8-2e2a7d68dc45 service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:17:32.739517 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0d8899-686d-49cf-98a8-2e2a7d68dc45 service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:17:32.740022 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0d8899-686d-49cf-98a8-2e2a7d68dc45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:17:32.743502 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-9c0d8899-686d-49cf-98a8-2e2a7d68dc45 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:17:32.935297 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0d8899-686d-49cf-98a8-2e2a7d68dc45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:17:32.936815 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0d8899-686d-49cf-98a8-2e2a7d68dc45 service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:17:32.958518 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 678/1373] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:17:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:17:33.681210 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19633172-0d35-404f-8591-2b580c8106fa None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:17:33.681989 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19633172-0d35-404f-8591-2b580c8106fa None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:17:33.681989 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19633172-0d35-404f-8591-2b580c8106fa None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:17:33.683281 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 696/1374] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:17:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:17:33.690129 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd1ac62-073b-46a2-9f8b-cd641ef9cddb None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:17:33.690592 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd1ac62-073b-46a2-9f8b-cd641ef9cddb None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:17:33.690674 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd1ac62-073b-46a2-9f8b-cd641ef9cddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:17:33.775157 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-3cd1ac62-073b-46a2-9f8b-cd641ef9cddb None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:17:33.776591 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-3cd1ac62-073b-46a2-9f8b-cd641ef9cddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:17:33.798437 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:17:33.894244 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 679/1375] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:17:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:17:33.954591 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19633172-0d35-404f-8591-2b580c8106fa None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:17:33.954591 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-19633172-0d35-404f-8591-2b580c8106fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:17:34.158648 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19633172-0d35-404f-8591-2b580c8106fa None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:17:34.249908 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:17:34.336080 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd0a4a5-690d-4623-9d2f-add5646b6d19 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a2ce1df87fa47e5be301916bf3245fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:17:34.336805 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd0a4a5-690d-4623-9d2f-add5646b6d19 service nova] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:17:34.337090 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd0a4a5-690d-4623-9d2f-add5646b6d19 service nova] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:17:34.337360 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd0a4a5-690d-4623-9d2f-add5646b6d19 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:17:34.339671 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-8bd0a4a5-690d-4623-9d2f-add5646b6d19 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:17:34.457407 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd0a4a5-690d-4623-9d2f-add5646b6d19 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:17:34.460022 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd0a4a5-690d-4623-9d2f-add5646b6d19 service nova] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:17:34.483758 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 697/1376] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:17:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 536 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:17:37.260629 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ddbdb9-fc69-4941-98de-1db664be7de5 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:17:37.261294 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ddbdb9-fc69-4941-98de-1db664be7de5 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:17:37.261845 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ddbdb9-fc69-4941-98de-1db664be7de5 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:17:37.264039 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 680/1377] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:17:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:17:37.274418 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e87993-e5a2-4dcd-b6b8-c0136cf5f8d6 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:17:37.275591 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e87993-e5a2-4dcd-b6b8-c0136cf5f8d6 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:17:37.276025 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e87993-e5a2-4dcd-b6b8-c0136cf5f8d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:17:37.415055 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-e0e87993-e5a2-4dcd-b6b8-c0136cf5f8d6 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:17:37.416417 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-e0e87993-e5a2-4dcd-b6b8-c0136cf5f8d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:17:37.430274 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:17:37.614568 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 698/1378] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:17:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:17:37.658399 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ddbdb9-fc69-4941-98de-1db664be7de5 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:17:37.659963 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-05ddbdb9-fc69-4941-98de-1db664be7de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:17:37.934879 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ddbdb9-fc69-4941-98de-1db664be7de5 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:17:37.941333 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:17:38.105171 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa6a92f-7c28-4f13-b317-b45fa92dcadb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a2ce1df87fa47e5be301916bf3245fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:17:38.106427 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa6a92f-7c28-4f13-b317-b45fa92dcadb service nova] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:17:38.107249 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa6a92f-7c28-4f13-b317-b45fa92dcadb service nova] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:17:38.107844 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa6a92f-7c28-4f13-b317-b45fa92dcadb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:17:38.110035 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-2aa6a92f-7c28-4f13-b317-b45fa92dcadb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:17:38.261221 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa6a92f-7c28-4f13-b317-b45fa92dcadb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:17:38.264399 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa6a92f-7c28-4f13-b317-b45fa92dcadb service nova] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:17:38.295543 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 681/1379] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:17:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 637 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:17:40.790848 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3d479a-74fa-44d8-a04c-59d9bf952477 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:17:40.791669 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3d479a-74fa-44d8-a04c-59d9bf952477 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:17:40.791669 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3d479a-74fa-44d8-a04c-59d9bf952477 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:17:40.792937 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 699/1380] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:17:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 21:17:40.798599 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2649f4-da57-40ba-a0b0-25e26be4a2d7 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:17:40.799606 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2649f4-da57-40ba-a0b0-25e26be4a2d7 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:17:40.799899 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2649f4-da57-40ba-a0b0-25e26be4a2d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:17:40.815054 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:17:40.893582 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-2a2649f4-da57-40ba-a0b0-25e26be4a2d7 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:17:40.895624 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-2a2649f4-da57-40ba-a0b0-25e26be4a2d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:17:41.051858 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 682/1381] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:17:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:17:41.104380 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3d479a-74fa-44d8-a04c-59d9bf952477 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:17:41.105961 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-1d3d479a-74fa-44d8-a04c-59d9bf952477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:17:41.355884 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:17:41.375093 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3d479a-74fa-44d8-a04c-59d9bf952477 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:17:41.584398 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d9ac49-227a-4ad0-a526-dbafa057edb6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a2ce1df87fa47e5be301916bf3245fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:17:41.585395 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d9ac49-227a-4ad0-a526-dbafa057edb6 service nova] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:17:41.585859 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d9ac49-227a-4ad0-a526-dbafa057edb6 service nova] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:17:41.586280 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d9ac49-227a-4ad0-a526-dbafa057edb6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:17:41.589354 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-14d9ac49-227a-4ad0-a526-dbafa057edb6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:17:41.753054 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d9ac49-227a-4ad0-a526-dbafa057edb6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:17:41.755942 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d9ac49-227a-4ad0-a526-dbafa057edb6 service nova] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:17:41.776012 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:17:41.793459 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 700/1382] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:17:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 689 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:17:44.602797 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6364db96-e22a-420c-a17a-1c9020c3e9cf None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:17:44.604452 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6364db96-e22a-420c-a17a-1c9020c3e9cf None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:17:44.605155 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6364db96-e22a-420c-a17a-1c9020c3e9cf None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:17:44.609446 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 683/1383] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:17:44 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:17:44.620927 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b60fce-0c2b-446a-8913-9f371818cbb4 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:17:44.621979 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b60fce-0c2b-446a-8913-9f371818cbb4 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:17:44.621979 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b60fce-0c2b-446a-8913-9f371818cbb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:17:44.726250 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-c3b60fce-0c2b-446a-8913-9f371818cbb4 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:17:44.727727 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-c3b60fce-0c2b-446a-8913-9f371818cbb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:17:44.891006 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 701/1384] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:17:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:17:44.952444 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6364db96-e22a-420c-a17a-1c9020c3e9cf None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:17:44.954023 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-6364db96-e22a-420c-a17a-1c9020c3e9cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:17:45.076337 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:17:45.190276 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6364db96-e22a-420c-a17a-1c9020c3e9cf None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:17:45.342126 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b953ae65-44aa-41c8-8f53-fff889ada4f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a2ce1df87fa47e5be301916bf3245fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:17:45.342896 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b953ae65-44aa-41c8-8f53-fff889ada4f1 service nova] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:17:45.343300 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b953ae65-44aa-41c8-8f53-fff889ada4f1 service nova] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:17:45.343683 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b953ae65-44aa-41c8-8f53-fff889ada4f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:17:45.346583 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-b953ae65-44aa-41c8-8f53-fff889ada4f1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:17:45.416007 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:17:45.478765 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b953ae65-44aa-41c8-8f53-fff889ada4f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:17:45.481234 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b953ae65-44aa-41c8-8f53-fff889ada4f1 service nova] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:17:45.520674 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 684/1385] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:17:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 569 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:17:47.753665 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6e318c-110a-4e14-86b3-f8f8a79a4dff None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:17:47.787452 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6e318c-110a-4e14-86b3-f8f8a79a4dff None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:17:47.787452 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6e318c-110a-4e14-86b3-f8f8a79a4dff None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:17:47.787452 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 702/1386] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:17:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 21:17:47.788532 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1ba920-d4f1-4ec1-acdf-ce5efc4890b0 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:17:47.788532 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1ba920-d4f1-4ec1-acdf-ce5efc4890b0 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:17:47.788532 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1ba920-d4f1-4ec1-acdf-ce5efc4890b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:17:47.856081 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-4f1ba920-d4f1-4ec1-acdf-ce5efc4890b0 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:17:47.857338 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-4f1ba920-d4f1-4ec1-acdf-ce5efc4890b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:17:47.992037 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 685/1387] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:17:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:17:48.040182 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6e318c-110a-4e14-86b3-f8f8a79a4dff None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:17:48.041758 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-1c6e318c-110a-4e14-86b3-f8f8a79a4dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:17:48.129871 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:17:48.221471 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6e318c-110a-4e14-86b3-f8f8a79a4dff None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:17:48.362169 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10f5d85-e39c-46ce-bd19-2371ac911095 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a2ce1df87fa47e5be301916bf3245fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:17:48.362718 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f5d85-e39c-46ce-bd19-2371ac911095 service nova] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:17:48.363000 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f5d85-e39c-46ce-bd19-2371ac911095 service nova] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:17:48.363273 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f5d85-e39c-46ce-bd19-2371ac911095 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:17:48.365195 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-a10f5d85-e39c-46ce-bd19-2371ac911095 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:17:48.425176 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:17:48.481839 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10f5d85-e39c-46ce-bd19-2371ac911095 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:17:48.483539 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10f5d85-e39c-46ce-bd19-2371ac911095 service nova] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:17:48.503807 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 703/1388] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:17:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:17:50.296824 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b8a39e-fb08-420b-84fa-15389f0e16f7 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:17:50.297380 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b8a39e-fb08-420b-84fa-15389f0e16f7 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:17:50.297380 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b8a39e-fb08-420b-84fa-15389f0e16f7 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:17:50.298838 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 686/1389] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:17:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:17:50.304972 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39dcc26-f050-4808-a920-7ea72f1aa02f None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:17:50.305496 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39dcc26-f050-4808-a920-7ea72f1aa02f None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:17:50.305496 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39dcc26-f050-4808-a920-7ea72f1aa02f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:17:50.398686 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-e39dcc26-f050-4808-a920-7ea72f1aa02f None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:17:50.400563 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-e39dcc26-f050-4808-a920-7ea72f1aa02f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:17:50.554575 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 704/1390] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:17:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:17:50.623812 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf907d-02ef-4014-8c6f-9c4492a36968 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:17:50.624067 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf907d-02ef-4014-8c6f-9c4492a36968 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:17:50.624349 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf907d-02ef-4014-8c6f-9c4492a36968 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:17:50.625719 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 687/1391] 10.222.0.25 () {54 vars in 966 bytes} [Thu Aug 4 21:17:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:17:50.631002 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e88f91-7756-4387-ab84-65ccd9493e9e None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:17:50.631290 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e88f91-7756-4387-ab84-65ccd9493e9e None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:17:50.631592 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e88f91-7756-4387-ab84-65ccd9493e9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:17:50.650516 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:17:50.739009 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-81e88f91-7756-4387-ab84-65ccd9493e9e None None] MFA Rules not processed for user `f6885c7d929f4f8e96d3890724c3b53f`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:17:50.740575 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-81e88f91-7756-4387-ab84-65ccd9493e9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:17:50.832903 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 705/1392] 10.222.0.25 () {58 vars in 1063 bytes} [Thu Aug 4 21:17:50 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:17:50.841522 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bf907d-02ef-4014-8c6f-9c4492a36968 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:17:50.843026 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-23bf907d-02ef-4014-8c6f-9c4492a36968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:17:50.878079 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:17:50.978268 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bf907d-02ef-4014-8c6f-9c4492a36968 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:17:51.003014 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96929470-f559-4d60-88e9-eb8fa961f0c2 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:17:51.003419 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96929470-f559-4d60-88e9-eb8fa961f0c2 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:17:51.003773 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96929470-f559-4d60-88e9-eb8fa961f0c2 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:17:51.005699 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 706/1393] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:17:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:17:51.014963 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0c584c-adc7-4276-b20d-6de80a239974 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:17:51.015576 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0c584c-adc7-4276-b20d-6de80a239974 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:17:51.016108 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0c584c-adc7-4276-b20d-6de80a239974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:17:51.091013 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b68291f-56fe-4768-aea4-c5479f08bc0d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6885c7d929f4f8e96d3890724c3b53f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:17:51.091590 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68291f-56fe-4768-aea4-c5479f08bc0d service glance] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:17:51.091951 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68291f-56fe-4768-aea4-c5479f08bc0d service glance] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:17:51.092225 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68291f-56fe-4768-aea4-c5479f08bc0d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:17:51.094009 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-2b68291f-56fe-4768-aea4-c5479f08bc0d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:17:51.102198 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-6e0c584c-adc7-4276-b20d-6de80a239974 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:17:51.103591 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-6e0c584c-adc7-4276-b20d-6de80a239974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:17:51.186342 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:17:51.205862 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:17:51.244032 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 707/1394] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:17:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:17:51.248640 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b68291f-56fe-4768-aea4-c5479f08bc0d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:17:51.253426 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b68291f-56fe-4768-aea4-c5479f08bc0d service glance] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:17:51.280589 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 688/1395] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:17:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:17:51.296900 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0c584c-adc7-4276-b20d-6de80a239974 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:17:51.298937 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-6e0c584c-adc7-4276-b20d-6de80a239974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:17:51.484922 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0c584c-adc7-4276-b20d-6de80a239974 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:17:51.559263 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:17:51.622868 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686e5d95-d2e5-42af-bf72-cc25a4d35fb6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a2ce1df87fa47e5be301916bf3245fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:17:51.623914 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686e5d95-d2e5-42af-bf72-cc25a4d35fb6 service nova] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:17:51.624410 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686e5d95-d2e5-42af-bf72-cc25a4d35fb6 service nova] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:17:51.624817 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686e5d95-d2e5-42af-bf72-cc25a4d35fb6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:17:51.628169 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-686e5d95-d2e5-42af-bf72-cc25a4d35fb6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:17:51.752394 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686e5d95-d2e5-42af-bf72-cc25a4d35fb6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:17:51.753921 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686e5d95-d2e5-42af-bf72-cc25a4d35fb6 service nova] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:17:51.773575 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 708/1396] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:17:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:17:52.367203 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb22e16-9c31-4276-b35c-3bf9c16fcbb8 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:17:52.367865 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb22e16-9c31-4276-b35c-3bf9c16fcbb8 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:17:52.367865 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb22e16-9c31-4276-b35c-3bf9c16fcbb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:17:52.445581 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-2cb22e16-9c31-4276-b35c-3bf9c16fcbb8 None None] MFA Rules not processed for user `23ea380385f649628cc27e3603bc700c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:17:52.447700 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-2cb22e16-9c31-4276-b35c-3bf9c16fcbb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:17:52.586235 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 689/1397] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Aug 4 21:17:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:17:52.644723 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5442685-8a6e-4acf-a725-06c65e7aa545 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:17:52.645268 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5442685-8a6e-4acf-a725-06c65e7aa545 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:17:52.645413 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5442685-8a6e-4acf-a725-06c65e7aa545 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:17:52.646880 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 709/1398] 10.222.0.25 () {54 vars in 943 bytes} [Thu Aug 4 21:17:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:17:52.656260 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff420b38-45c8-4c8c-8cce-44983525f493 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:17:52.656709 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff420b38-45c8-4c8c-8cce-44983525f493 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:17:52.657058 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff420b38-45c8-4c8c-8cce-44983525f493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:17:52.742082 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:17:52.767853 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-ff420b38-45c8-4c8c-8cce-44983525f493 None None] MFA Rules not processed for user `58935367c0ad445f8477cdac0df04d0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:17:52.769231 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-ff420b38-45c8-4c8c-8cce-44983525f493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:17:52.925866 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 690/1399] 10.222.0.25 () {58 vars in 1040 bytes} [Thu Aug 4 21:17:52 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:17:52.943716 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5442685-8a6e-4acf-a725-06c65e7aa545 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:17:52.945369 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-e5442685-8a6e-4acf-a725-06c65e7aa545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:17:53.092475 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:17:53.173783 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5442685-8a6e-4acf-a725-06c65e7aa545 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:17:53.320945 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5409364f-fb36-42fe-9359-35f50fd58e1e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58935367c0ad445f8477cdac0df04d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:17:53.322194 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5409364f-fb36-42fe-9359-35f50fd58e1e service swift] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:17:53.322480 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5409364f-fb36-42fe-9359-35f50fd58e1e service swift] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:17:53.322862 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5409364f-fb36-42fe-9359-35f50fd58e1e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:17:53.325823 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-5409364f-fb36-42fe-9359-35f50fd58e1e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:17:53.408219 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:17:53.435219 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5409364f-fb36-42fe-9359-35f50fd58e1e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:17:53.437150 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5409364f-fb36-42fe-9359-35f50fd58e1e service swift] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:17:53.450130 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 710/1400] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Aug 4 21:17:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 21:17:54.365391 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40cc544-6419-4da6-912e-3c97cb491afb None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:17:54.366580 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40cc544-6419-4da6-912e-3c97cb491afb None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:17:54.367138 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40cc544-6419-4da6-912e-3c97cb491afb None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:17:54.369150 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 691/1401] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:17:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:17:54.376001 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6a5176-94c0-4dca-88ca-bead8d9b9cc0 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:17:54.376828 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6a5176-94c0-4dca-88ca-bead8d9b9cc0 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:17:54.377331 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6a5176-94c0-4dca-88ca-bead8d9b9cc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:17:54.458875 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-5a6a5176-94c0-4dca-88ca-bead8d9b9cc0 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:17:54.460166 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-5a6a5176-94c0-4dca-88ca-bead8d9b9cc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:17:54.595401 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 711/1402] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:17:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:17:54.671094 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40cc544-6419-4da6-912e-3c97cb491afb None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:17:54.687608 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-a40cc544-6419-4da6-912e-3c97cb491afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:17:54.864952 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:17:54.922320 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40cc544-6419-4da6-912e-3c97cb491afb None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:17:55.090131 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf2a43e-1160-43e3-a86b-e7e4700ea116 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a2ce1df87fa47e5be301916bf3245fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:17:55.091132 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf2a43e-1160-43e3-a86b-e7e4700ea116 service nova] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:17:55.091630 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf2a43e-1160-43e3-a86b-e7e4700ea116 service nova] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:17:55.091916 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf2a43e-1160-43e3-a86b-e7e4700ea116 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:17:55.093654 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-baf2a43e-1160-43e3-a86b-e7e4700ea116 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:17:55.256286 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf2a43e-1160-43e3-a86b-e7e4700ea116 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:17:55.258194 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf2a43e-1160-43e3-a86b-e7e4700ea116 service nova] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:17:55.282045 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 692/1403] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:17:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 611 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:17:55.556206 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:17:57.999611 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bb9f3c-a74c-4668-9bf1-5a6ef81b7466 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:17:57.999611 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bb9f3c-a74c-4668-9bf1-5a6ef81b7466 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:17:57.999611 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bb9f3c-a74c-4668-9bf1-5a6ef81b7466 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:17:58.000392 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 712/1404] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:17:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 21:17:58.011054 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9141cb4-fe89-4630-8bcc-746724edc58a None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:17:58.011673 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9141cb4-fe89-4630-8bcc-746724edc58a None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:17:58.012095 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9141cb4-fe89-4630-8bcc-746724edc58a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:17:58.116953 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-b9141cb4-fe89-4630-8bcc-746724edc58a None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:17:58.118840 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-b9141cb4-fe89-4630-8bcc-746724edc58a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:17:58.261691 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 693/1405] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:17:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:17:58.309908 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bb9f3c-a74c-4668-9bf1-5a6ef81b7466 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:17:58.312713 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-f5bb9f3c-a74c-4668-9bf1-5a6ef81b7466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:17:58.424305 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:17:58.575437 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bb9f3c-a74c-4668-9bf1-5a6ef81b7466 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:17:58.782749 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2037f1-46fc-4f65-b9d1-bbcea023c5ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a2ce1df87fa47e5be301916bf3245fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:17:58.783970 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2037f1-46fc-4f65-b9d1-bbcea023c5ef service nova] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:17:58.784482 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2037f1-46fc-4f65-b9d1-bbcea023c5ef service nova] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:17:58.784975 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2037f1-46fc-4f65-b9d1-bbcea023c5ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:17:58.788328 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-5d2037f1-46fc-4f65-b9d1-bbcea023c5ef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:17:58.816908 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:17:58.942785 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2037f1-46fc-4f65-b9d1-bbcea023c5ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:17:58.944595 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2037f1-46fc-4f65-b9d1-bbcea023c5ef service nova] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:17:58.972998 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 713/1406] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:17:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 664 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:18:01.569444 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f777c0dd-6823-49b4-9d37-2bee5fcd45a6 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:18:01.569444 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f777c0dd-6823-49b4-9d37-2bee5fcd45a6 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:18:01.569444 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f777c0dd-6823-49b4-9d37-2bee5fcd45a6 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:18:01.569444 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 694/1407] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:18:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:18:01.570573 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118a998b-a407-4bf6-9832-cfd411b49b3f None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:18:01.571276 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118a998b-a407-4bf6-9832-cfd411b49b3f None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:18:01.571276 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118a998b-a407-4bf6-9832-cfd411b49b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:18:01.674107 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-118a998b-a407-4bf6-9832-cfd411b49b3f None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:18:01.675853 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-118a998b-a407-4bf6-9832-cfd411b49b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:18:01.815456 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:18:01.815456 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 714/1408] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:18:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:18:01.869184 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f777c0dd-6823-49b4-9d37-2bee5fcd45a6 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:18:01.870725 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-f777c0dd-6823-49b4-9d37-2bee5fcd45a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:18:01.930536 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:18:02.101926 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f777c0dd-6823-49b4-9d37-2bee5fcd45a6 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:18:02.276852 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b711f5ed-84d9-44f9-891b-1b13d4b58744 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a2ce1df87fa47e5be301916bf3245fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:18:02.277576 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b711f5ed-84d9-44f9-891b-1b13d4b58744 service nova] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:18:02.277857 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b711f5ed-84d9-44f9-891b-1b13d4b58744 service nova] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:18:02.278578 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b711f5ed-84d9-44f9-891b-1b13d4b58744 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:18:02.281295 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-b711f5ed-84d9-44f9-891b-1b13d4b58744 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:18:02.291521 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:18:02.417998 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b711f5ed-84d9-44f9-891b-1b13d4b58744 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:18:02.419683 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b711f5ed-84d9-44f9-891b-1b13d4b58744 service nova] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:18:02.449306 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 695/1409] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:18:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 581 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:18:11.870384 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=2 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:18:12.445480 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=42 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:18:20.230113 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2424d8f6-1516-428b-b64d-b9eee4b7a06e None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:18:20.230578 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2424d8f6-1516-428b-b64d-b9eee4b7a06e None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:18:20.231111 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2424d8f6-1516-428b-b64d-b9eee4b7a06e None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:18:20.233601 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 715/1410] 10.222.0.25 () {54 vars in 918 bytes} [Thu Aug 4 21:18:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:18:20.240728 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b8c12a-94ef-41d8-bd4e-802bb00c93bf None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:18:20.242315 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b8c12a-94ef-41d8-bd4e-802bb00c93bf None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:18:20.242315 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b8c12a-94ef-41d8-bd4e-802bb00c93bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:18:20.337434 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-e8b8c12a-94ef-41d8-bd4e-802bb00c93bf None None] MFA Rules not processed for user `67d44a0098ab4c36b14b36e9e59ae75c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:18:20.338660 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-e8b8c12a-94ef-41d8-bd4e-802bb00c93bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:18:20.489656 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 696/1411] 10.222.0.25 () {58 vars in 1015 bytes} [Thu Aug 4 21:18:20 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:18:20.508348 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891116a4-14ea-4266-aab0-d90e2fe16497 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:18:20.508788 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891116a4-14ea-4266-aab0-d90e2fe16497 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:18:20.509103 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891116a4-14ea-4266-aab0-d90e2fe16497 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:18:20.510643 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 716/1412] 10.222.0.25 () {54 vars in 918 bytes} [Thu Aug 4 21:18:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:18:20.517031 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0442ff-97a1-464d-a480-e8ce5535d68d None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:18:20.517408 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0442ff-97a1-464d-a480-e8ce5535d68d None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:18:20.517798 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0442ff-97a1-464d-a480-e8ce5535d68d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:18:20.622169 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:18:20.626392 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-3a0442ff-97a1-464d-a480-e8ce5535d68d None None] MFA Rules not processed for user `6a2ce1df87fa47e5be301916bf3245fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:18:20.628160 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-3a0442ff-97a1-464d-a480-e8ce5535d68d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:18:20.811757 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 697/1413] 10.222.0.25 () {58 vars in 1015 bytes} [Thu Aug 4 21:18:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:18:23.555680 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1596c5-a12a-4ea0-b3b4-7bf6a276a4b5 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:18:23.556483 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1596c5-a12a-4ea0-b3b4-7bf6a276a4b5 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:18:23.556483 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1596c5-a12a-4ea0-b3b4-7bf6a276a4b5 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:18:23.557575 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 717/1414] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 21:18:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:18:23.564660 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0f4803-7405-4a6f-bcb0-92cbd3280bf3 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:18:23.565402 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0f4803-7405-4a6f-bcb0-92cbd3280bf3 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:18:23.565719 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0f4803-7405-4a6f-bcb0-92cbd3280bf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:18:23.661389 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-4b0f4803-7405-4a6f-bcb0-92cbd3280bf3 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:18:23.662752 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-4b0f4803-7405-4a6f-bcb0-92cbd3280bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:18:23.742062 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:18:23.791827 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 698/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 21:18:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:18:23.844416 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1596c5-a12a-4ea0-b3b4-7bf6a276a4b5 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:18:23.846047 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-0a1596c5-a12a-4ea0-b3b4-7bf6a276a4b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:18:24.039745 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1596c5-a12a-4ea0-b3b4-7bf6a276a4b5 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:18:24.140465 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:18:24.196942 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6fe1b8-22fd-4e32-96bf-69f4fcd4a2ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a2ce1df87fa47e5be301916bf3245fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:18:24.197723 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6fe1b8-22fd-4e32-96bf-69f4fcd4a2ac service nova] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:18:24.198069 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6fe1b8-22fd-4e32-96bf-69f4fcd4a2ac service nova] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:18:24.198384 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6fe1b8-22fd-4e32-96bf-69f4fcd4a2ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:18:24.200371 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-8c6fe1b8-22fd-4e32-96bf-69f4fcd4a2ac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:18:24.330090 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6fe1b8-22fd-4e32-96bf-69f4fcd4a2ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:18:24.331779 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6fe1b8-22fd-4e32-96bf-69f4fcd4a2ac service nova] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:18:24.357010 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 718/1416] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:18:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:18:25.373611 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b675f6f-cb22-427a-93f8-eff50e6768a1 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:18:25.374183 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b675f6f-cb22-427a-93f8-eff50e6768a1 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:18:25.374183 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b675f6f-cb22-427a-93f8-eff50e6768a1 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:18:25.375707 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 699/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 21:18:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:18:25.381673 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bcc378-49ca-4e66-bb38-e96853998eab None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:18:25.382225 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bcc378-49ca-4e66-bb38-e96853998eab None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:18:25.382225 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bcc378-49ca-4e66-bb38-e96853998eab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:18:25.495630 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-12bcc378-49ca-4e66-bb38-e96853998eab None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:18:25.496571 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-12bcc378-49ca-4e66-bb38-e96853998eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:18:25.528524 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:18:25.620408 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 719/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 21:18:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:18:25.787029 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b675f6f-cb22-427a-93f8-eff50e6768a1 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:18:25.788453 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-2b675f6f-cb22-427a-93f8-eff50e6768a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:18:26.006871 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b675f6f-cb22-427a-93f8-eff50e6768a1 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:18:26.054143 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:18:26.171935 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2a4a28-49df-4c39-a996-a3333b78aa7e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a2ce1df87fa47e5be301916bf3245fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:18:26.172577 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2a4a28-49df-4c39-a996-a3333b78aa7e service nova] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:18:26.172859 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2a4a28-49df-4c39-a996-a3333b78aa7e service nova] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:18:26.173132 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2a4a28-49df-4c39-a996-a3333b78aa7e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:18:26.174896 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-7f2a4a28-49df-4c39-a996-a3333b78aa7e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:18:26.285464 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2a4a28-49df-4c39-a996-a3333b78aa7e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:18:26.287090 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2a4a28-49df-4c39-a996-a3333b78aa7e service nova] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:18:26.309886 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 700/1419] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:18:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:18:27.317860 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1df728d-6c9b-485e-8f0a-cad463199bc2 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:18:27.318680 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1df728d-6c9b-485e-8f0a-cad463199bc2 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:18:27.318680 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1df728d-6c9b-485e-8f0a-cad463199bc2 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:18:27.319883 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 720/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 21:18:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:18:27.326642 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c8db3c-938f-4905-ab8e-89dc3b3a47f2 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:18:27.327519 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c8db3c-938f-4905-ab8e-89dc3b3a47f2 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:18:27.327680 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c8db3c-938f-4905-ab8e-89dc3b3a47f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:18:27.400672 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:18:27.443196 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-01c8db3c-938f-4905-ab8e-89dc3b3a47f2 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:18:27.444561 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-01c8db3c-938f-4905-ab8e-89dc3b3a47f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:18:27.572209 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 701/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 21:18:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:18:27.745751 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1df728d-6c9b-485e-8f0a-cad463199bc2 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:18:27.747414 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-c1df728d-6c9b-485e-8f0a-cad463199bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:18:27.934401 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1df728d-6c9b-485e-8f0a-cad463199bc2 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:18:27.952667 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:18:28.107432 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59d974e-4bc5-47bb-8d85-0f729d1a8a83 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a2ce1df87fa47e5be301916bf3245fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:18:28.108379 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59d974e-4bc5-47bb-8d85-0f729d1a8a83 service nova] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:18:28.108803 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59d974e-4bc5-47bb-8d85-0f729d1a8a83 service nova] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:18:28.109210 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59d974e-4bc5-47bb-8d85-0f729d1a8a83 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:18:28.112153 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-e59d974e-4bc5-47bb-8d85-0f729d1a8a83 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:18:28.272165 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59d974e-4bc5-47bb-8d85-0f729d1a8a83 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:18:28.273939 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59d974e-4bc5-47bb-8d85-0f729d1a8a83 service nova] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:18:28.299153 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 721/1422] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:18:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 554 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:18:29.344425 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25012f15-6042-430f-adad-c3bc1473277c None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:18:29.345261 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25012f15-6042-430f-adad-c3bc1473277c None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:18:29.345261 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25012f15-6042-430f-adad-c3bc1473277c None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:18:29.346328 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 702/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 21:18:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:18:29.352505 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709fe853-bebd-4fd4-9886-b30c03fdd9f7 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:18:29.354967 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709fe853-bebd-4fd4-9886-b30c03fdd9f7 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:18:29.354967 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709fe853-bebd-4fd4-9886-b30c03fdd9f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:18:29.379919 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:18:29.460153 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-709fe853-bebd-4fd4-9886-b30c03fdd9f7 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:18:29.461816 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-709fe853-bebd-4fd4-9886-b30c03fdd9f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:18:29.605998 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 722/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 21:18:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:18:29.818210 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25012f15-6042-430f-adad-c3bc1473277c None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:18:29.819887 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-25012f15-6042-430f-adad-c3bc1473277c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:18:29.977681 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:18:30.024904 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25012f15-6042-430f-adad-c3bc1473277c None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:18:30.200422 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f083c5-a38f-4a20-8ea8-42f5513f9e5c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a2ce1df87fa47e5be301916bf3245fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:18:30.201375 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f083c5-a38f-4a20-8ea8-42f5513f9e5c service nova] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:18:30.201842 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f083c5-a38f-4a20-8ea8-42f5513f9e5c service nova] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:18:30.202127 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f083c5-a38f-4a20-8ea8-42f5513f9e5c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:18:30.204212 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-17f083c5-a38f-4a20-8ea8-42f5513f9e5c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:18:30.306368 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:18:30.326746 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f083c5-a38f-4a20-8ea8-42f5513f9e5c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:18:30.329186 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f083c5-a38f-4a20-8ea8-42f5513f9e5c service nova] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:18:30.360238 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 703/1425] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:18:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 540 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:18:31.305841 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4abe0ef-79f1-4307-9b6b-6b3ae1b29f72 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:18:31.306426 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4abe0ef-79f1-4307-9b6b-6b3ae1b29f72 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:18:31.306426 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4abe0ef-79f1-4307-9b6b-6b3ae1b29f72 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:18:31.307794 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 723/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 21:18:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:18:31.313663 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257f35b6-0c4d-4946-9009-7f57f8a1cc0d None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:18:31.313948 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257f35b6-0c4d-4946-9009-7f57f8a1cc0d None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:18:31.314225 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257f35b6-0c4d-4946-9009-7f57f8a1cc0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:18:31.409390 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-257f35b6-0c4d-4946-9009-7f57f8a1cc0d None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:18:31.410710 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-257f35b6-0c4d-4946-9009-7f57f8a1cc0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:18:31.521193 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 704/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 21:18:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:18:31.673690 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4abe0ef-79f1-4307-9b6b-6b3ae1b29f72 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:18:31.675299 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-d4abe0ef-79f1-4307-9b6b-6b3ae1b29f72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:18:31.783890 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:18:31.856220 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4abe0ef-79f1-4307-9b6b-6b3ae1b29f72 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:18:32.012993 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aee677-5e48-4fbb-9a41-40bf78e89fcf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a2ce1df87fa47e5be301916bf3245fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:18:32.013658 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aee677-5e48-4fbb-9a41-40bf78e89fcf service nova] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:18:32.013986 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aee677-5e48-4fbb-9a41-40bf78e89fcf service nova] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:18:32.014268 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aee677-5e48-4fbb-9a41-40bf78e89fcf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:18:32.016128 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-10aee677-5e48-4fbb-9a41-40bf78e89fcf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:18:32.073435 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:18:32.139697 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10aee677-5e48-4fbb-9a41-40bf78e89fcf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:18:32.141336 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10aee677-5e48-4fbb-9a41-40bf78e89fcf service nova] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:18:32.161970 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 724/1428] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:18:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:18:33.124509 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ee14d0-52aa-438b-9148-5b3371121fbb None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:18:33.125082 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ee14d0-52aa-438b-9148-5b3371121fbb None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:18:33.125082 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ee14d0-52aa-438b-9148-5b3371121fbb None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:18:33.126459 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 705/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 21:18:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:18:33.132570 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82a37cf-e25d-4073-869a-37109d028b1d None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:18:33.133111 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82a37cf-e25d-4073-869a-37109d028b1d None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:18:33.133111 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82a37cf-e25d-4073-869a-37109d028b1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:18:33.223808 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-b82a37cf-e25d-4073-869a-37109d028b1d None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:18:33.225036 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-b82a37cf-e25d-4073-869a-37109d028b1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:18:33.337070 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 725/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 21:18:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:18:33.497041 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ee14d0-52aa-438b-9148-5b3371121fbb None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:18:33.498601 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-a5ee14d0-52aa-438b-9148-5b3371121fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:18:33.566270 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:18:33.670312 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ee14d0-52aa-438b-9148-5b3371121fbb None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:18:33.807100 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ef4ab0-2163-4497-b739-48bd11348bb0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a2ce1df87fa47e5be301916bf3245fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:18:33.807808 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ef4ab0-2163-4497-b739-48bd11348bb0 service nova] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:18:33.808087 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ef4ab0-2163-4497-b739-48bd11348bb0 service nova] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:18:33.808364 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ef4ab0-2163-4497-b739-48bd11348bb0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:18:33.810057 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-c9ef4ab0-2163-4497-b739-48bd11348bb0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:18:33.817483 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:18:33.903700 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ef4ab0-2163-4497-b739-48bd11348bb0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:18:33.905336 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ef4ab0-2163-4497-b739-48bd11348bb0 service nova] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:18:33.925172 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 706/1431] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:18:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:18:34.892225 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f453640e-057c-4b33-91cd-095824d3d055 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:18:34.892818 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f453640e-057c-4b33-91cd-095824d3d055 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:18:34.892818 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f453640e-057c-4b33-91cd-095824d3d055 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:18:34.894131 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 726/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 21:18:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:18:34.899190 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c75352-bbc8-4675-893c-ed7e3f739629 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:18:34.899706 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c75352-bbc8-4675-893c-ed7e3f739629 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:18:34.899828 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c75352-bbc8-4675-893c-ed7e3f739629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:18:34.981759 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-23c75352-bbc8-4675-893c-ed7e3f739629 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:18:34.982969 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-23c75352-bbc8-4675-893c-ed7e3f739629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:18:35.052554 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:18:35.082767 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 707/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 21:18:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:18:35.236004 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f453640e-057c-4b33-91cd-095824d3d055 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:18:35.237521 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-f453640e-057c-4b33-91cd-095824d3d055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:18:35.256964 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:18:35.404271 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f453640e-057c-4b33-91cd-095824d3d055 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:18:35.496368 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:18:35.525668 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ea4fa1-2bb7-4740-9db9-1465f2c7ac6e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a2ce1df87fa47e5be301916bf3245fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:18:35.526290 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ea4fa1-2bb7-4740-9db9-1465f2c7ac6e service nova] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:18:35.526568 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ea4fa1-2bb7-4740-9db9-1465f2c7ac6e service nova] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:18:35.526836 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ea4fa1-2bb7-4740-9db9-1465f2c7ac6e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:18:35.528523 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-52ea4fa1-2bb7-4740-9db9-1465f2c7ac6e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:18:35.632674 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ea4fa1-2bb7-4740-9db9-1465f2c7ac6e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:18:35.635018 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ea4fa1-2bb7-4740-9db9-1465f2c7ac6e service nova] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:18:35.657499 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 727/1434] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:18:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:18:36.623683 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa4aa3a-47d1-4ff6-aad5-4dc73c3428a6 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:18:36.624201 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa4aa3a-47d1-4ff6-aad5-4dc73c3428a6 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:18:36.624201 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa4aa3a-47d1-4ff6-aad5-4dc73c3428a6 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:18:36.625544 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 708/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 21:18:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:18:36.630849 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb7561-2b38-4497-88e5-85a820459a11 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:18:36.631389 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb7561-2b38-4497-88e5-85a820459a11 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:18:36.631509 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb7561-2b38-4497-88e5-85a820459a11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:18:36.713182 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-5fcb7561-2b38-4497-88e5-85a820459a11 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:18:36.714414 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-5fcb7561-2b38-4497-88e5-85a820459a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:18:36.760861 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:18:36.819471 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 728/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 21:18:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:18:36.981910 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa4aa3a-47d1-4ff6-aad5-4dc73c3428a6 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:18:36.983457 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-dfa4aa3a-47d1-4ff6-aad5-4dc73c3428a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:18:37.147781 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa4aa3a-47d1-4ff6-aad5-4dc73c3428a6 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:18:37.207776 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:18:37.271292 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf0ee9b-e334-4445-acb3-390bd5d2c664 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a2ce1df87fa47e5be301916bf3245fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:18:37.271878 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf0ee9b-e334-4445-acb3-390bd5d2c664 service nova] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:18:37.272157 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf0ee9b-e334-4445-acb3-390bd5d2c664 service nova] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:18:37.272429 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf0ee9b-e334-4445-acb3-390bd5d2c664 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:18:37.274111 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-3bf0ee9b-e334-4445-acb3-390bd5d2c664 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:18:37.378632 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf0ee9b-e334-4445-acb3-390bd5d2c664 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:18:37.380588 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf0ee9b-e334-4445-acb3-390bd5d2c664 service nova] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:18:37.402218 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 709/1437] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:18:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:18:38.350034 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e18785a-7e2e-4296-a7f4-d4decde6a03c None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:18:38.350623 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e18785a-7e2e-4296-a7f4-d4decde6a03c None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:18:38.350623 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e18785a-7e2e-4296-a7f4-d4decde6a03c None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:18:38.352113 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 729/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 21:18:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:18:38.357762 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103da7b7-b9e5-446f-915f-db01de550027 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:18:38.358193 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103da7b7-b9e5-446f-915f-db01de550027 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:18:38.358301 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103da7b7-b9e5-446f-915f-db01de550027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:18:38.443131 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-103da7b7-b9e5-446f-915f-db01de550027 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:18:38.444395 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-103da7b7-b9e5-446f-915f-db01de550027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:18:38.447049 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:18:38.562037 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 710/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 21:18:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:18:38.716622 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e18785a-7e2e-4296-a7f4-d4decde6a03c None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:18:38.718272 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-1e18785a-7e2e-4296-a7f4-d4decde6a03c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:18:38.918287 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e18785a-7e2e-4296-a7f4-d4decde6a03c None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:18:38.954147 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:18:39.048308 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6069d082-af77-4871-baa1-3a995d92b422 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a2ce1df87fa47e5be301916bf3245fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:18:39.048937 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6069d082-af77-4871-baa1-3a995d92b422 service nova] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:18:39.049226 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6069d082-af77-4871-baa1-3a995d92b422 service nova] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:18:39.049494 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6069d082-af77-4871-baa1-3a995d92b422 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:18:39.051130 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-6069d082-af77-4871-baa1-3a995d92b422 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:18:39.160582 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6069d082-af77-4871-baa1-3a995d92b422 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:18:39.162252 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6069d082-af77-4871-baa1-3a995d92b422 service nova] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:18:39.185543 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 730/1440] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:18:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:18:41.192416 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04259f4-7b6a-4d7c-a049-449c9848d308 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:18:41.200510 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04259f4-7b6a-4d7c-a049-449c9848d308 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:18:41.200510 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04259f4-7b6a-4d7c-a049-449c9848d308 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:18:41.200510 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 711/1441] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 21:18:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:18:41.200978 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de44596b-b2d5-4552-8b1c-03a4e6ed81ee None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:18:41.200978 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de44596b-b2d5-4552-8b1c-03a4e6ed81ee None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:18:41.200978 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de44596b-b2d5-4552-8b1c-03a4e6ed81ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:18:41.248893 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:18:41.285757 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-de44596b-b2d5-4552-8b1c-03a4e6ed81ee None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:18:41.286957 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-de44596b-b2d5-4552-8b1c-03a4e6ed81ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:18:41.407049 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 731/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 21:18:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:18:41.447800 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04259f4-7b6a-4d7c-a049-449c9848d308 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:18:41.449375 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-b04259f4-7b6a-4d7c-a049-449c9848d308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:18:41.568300 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04259f4-7b6a-4d7c-a049-449c9848d308 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:18:41.602903 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:18:41.660159 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3119ea-a32d-45bc-84f9-fda9e5ce7041 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:18:41.660797 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3119ea-a32d-45bc-84f9-fda9e5ce7041 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:18:41.661083 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3119ea-a32d-45bc-84f9-fda9e5ce7041 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:18:41.661362 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3119ea-a32d-45bc-84f9-fda9e5ce7041 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:18:41.663103 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-7a3119ea-a32d-45bc-84f9-fda9e5ce7041 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:18:41.783822 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3119ea-a32d-45bc-84f9-fda9e5ce7041 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:18:41.785203 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3119ea-a32d-45bc-84f9-fda9e5ce7041 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:18:41.806082 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 712/1443] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:18:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:18:44.108643 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd5ff26-6ed4-4c25-aa78-2345e6d45bf9 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:18:44.109221 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd5ff26-6ed4-4c25-aa78-2345e6d45bf9 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:18:44.109221 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd5ff26-6ed4-4c25-aa78-2345e6d45bf9 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:18:44.110580 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 732/1444] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 21:18:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:18:44.116357 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e71a4-cbcf-4e0d-abb5-cceed27bbb05 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:18:44.116961 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e71a4-cbcf-4e0d-abb5-cceed27bbb05 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:18:44.116961 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e71a4-cbcf-4e0d-abb5-cceed27bbb05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:18:44.198674 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-819e71a4-cbcf-4e0d-abb5-cceed27bbb05 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:18:44.199949 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-819e71a4-cbcf-4e0d-abb5-cceed27bbb05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:18:44.214252 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:18:44.310527 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 713/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 21:18:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:18:44.349278 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd5ff26-6ed4-4c25-aa78-2345e6d45bf9 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:18:44.350796 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-dfd5ff26-6ed4-4c25-aa78-2345e6d45bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:18:44.489090 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd5ff26-6ed4-4c25-aa78-2345e6d45bf9 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:18:44.504446 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:18:44.580746 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01f0517-307b-4e10-97f3-76d8ab2e9633 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:18:44.581404 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01f0517-307b-4e10-97f3-76d8ab2e9633 service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:18:44.581757 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01f0517-307b-4e10-97f3-76d8ab2e9633 service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:18:44.582038 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01f0517-307b-4e10-97f3-76d8ab2e9633 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:18:44.583892 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-f01f0517-307b-4e10-97f3-76d8ab2e9633 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:18:44.712554 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01f0517-307b-4e10-97f3-76d8ab2e9633 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:18:44.714143 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01f0517-307b-4e10-97f3-76d8ab2e9633 service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:18:44.734952 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 733/1446] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:18:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:18:47.377801 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7598b5eb-aa04-4864-8ab2-4c272335d69e None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:18:47.378330 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7598b5eb-aa04-4864-8ab2-4c272335d69e None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:18:47.378330 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7598b5eb-aa04-4864-8ab2-4c272335d69e None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:18:47.379667 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 714/1447] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 21:18:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:18:47.385083 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccdd15a-e9a2-491b-a669-38747e3cfdec None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:18:47.385562 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccdd15a-e9a2-491b-a669-38747e3cfdec None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:18:47.385671 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccdd15a-e9a2-491b-a669-38747e3cfdec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:18:47.435161 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:18:47.452820 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-dccdd15a-e9a2-491b-a669-38747e3cfdec None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:18:47.454027 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-dccdd15a-e9a2-491b-a669-38747e3cfdec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:18:47.553811 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 734/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 21:18:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:18:47.593155 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7598b5eb-aa04-4864-8ab2-4c272335d69e None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:18:47.594679 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-7598b5eb-aa04-4864-8ab2-4c272335d69e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:18:47.700559 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7598b5eb-aa04-4864-8ab2-4c272335d69e None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:18:47.742646 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:18:47.779865 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba86351b-1d24-4312-97da-53670b0abfaf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:18:47.780489 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba86351b-1d24-4312-97da-53670b0abfaf service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:18:47.780773 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba86351b-1d24-4312-97da-53670b0abfaf service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:18:47.781049 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba86351b-1d24-4312-97da-53670b0abfaf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:18:47.782721 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-ba86351b-1d24-4312-97da-53670b0abfaf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:18:47.908464 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba86351b-1d24-4312-97da-53670b0abfaf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:18:47.909948 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba86351b-1d24-4312-97da-53670b0abfaf service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:18:47.931063 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 715/1449] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:18:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:18:50.208267 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca073ab-1921-432b-8c30-60372a9a1904 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:18:50.208831 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca073ab-1921-432b-8c30-60372a9a1904 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:18:50.208831 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca073ab-1921-432b-8c30-60372a9a1904 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:18:50.210185 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 735/1450] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 21:18:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:18:50.216809 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3459ff61-1d51-42df-8521-fb591803490a None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:18:50.217276 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3459ff61-1d51-42df-8521-fb591803490a None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:18:50.217390 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3459ff61-1d51-42df-8521-fb591803490a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:18:50.302092 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-3459ff61-1d51-42df-8521-fb591803490a None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:18:50.303338 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-3459ff61-1d51-42df-8521-fb591803490a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:18:50.330635 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:18:50.438553 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 716/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 21:18:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:18:50.484585 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca073ab-1921-432b-8c30-60372a9a1904 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:18:50.486186 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-2ca073ab-1921-432b-8c30-60372a9a1904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:18:50.619461 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca073ab-1921-432b-8c30-60372a9a1904 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:18:50.653684 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:18:50.729475 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b848157-a78b-43ee-b753-a6e91088f25c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:18:50.730027 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b848157-a78b-43ee-b753-a6e91088f25c service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:18:50.730304 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b848157-a78b-43ee-b753-a6e91088f25c service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:18:50.730573 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b848157-a78b-43ee-b753-a6e91088f25c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:18:50.732344 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-3b848157-a78b-43ee-b753-a6e91088f25c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:18:50.859423 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b848157-a78b-43ee-b753-a6e91088f25c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:18:50.860895 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b848157-a78b-43ee-b753-a6e91088f25c service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:18:50.884477 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 736/1452] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:18:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:18:53.325984 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccbf959-2358-4ea4-9e4e-f17713b1a33b None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:18:53.326544 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccbf959-2358-4ea4-9e4e-f17713b1a33b None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:18:53.326658 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccbf959-2358-4ea4-9e4e-f17713b1a33b None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:18:53.328015 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 717/1453] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 21:18:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:18:53.333469 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76df51b9-b1fa-49ad-b4ca-d4e13978986e None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:18:53.333912 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76df51b9-b1fa-49ad-b4ca-d4e13978986e None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:18:53.334000 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76df51b9-b1fa-49ad-b4ca-d4e13978986e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:18:53.399960 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:18:53.402978 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-76df51b9-b1fa-49ad-b4ca-d4e13978986e None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:18:53.404244 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-76df51b9-b1fa-49ad-b4ca-d4e13978986e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:18:53.534211 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 737/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 21:18:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:18:53.574024 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccbf959-2358-4ea4-9e4e-f17713b1a33b None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:18:53.575603 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-3ccbf959-2358-4ea4-9e4e-f17713b1a33b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:18:53.696213 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccbf959-2358-4ea4-9e4e-f17713b1a33b None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:18:53.755176 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:18:53.787273 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94459f1b-1656-411a-929a-0f65dd79f381 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:18:53.788356 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94459f1b-1656-411a-929a-0f65dd79f381 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:18:53.788773 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94459f1b-1656-411a-929a-0f65dd79f381 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:18:53.789249 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94459f1b-1656-411a-929a-0f65dd79f381 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:18:53.792603 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-94459f1b-1656-411a-929a-0f65dd79f381 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:18:53.929016 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94459f1b-1656-411a-929a-0f65dd79f381 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:18:53.932967 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94459f1b-1656-411a-929a-0f65dd79f381 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:18:53.956476 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 718/1455] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:18:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:18:56.148132 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28e01c3-4fda-4aea-ae15-7d0f73ea11dd None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:18:56.148741 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28e01c3-4fda-4aea-ae15-7d0f73ea11dd None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:18:56.148741 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28e01c3-4fda-4aea-ae15-7d0f73ea11dd None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:18:56.150093 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 738/1456] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 21:18:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:18:56.158348 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a4aceb-2193-459d-ac82-faaa2d42a27c None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:18:56.158909 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a4aceb-2193-459d-ac82-faaa2d42a27c None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:18:56.159292 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a4aceb-2193-459d-ac82-faaa2d42a27c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:18:56.284335 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-e7a4aceb-2193-459d-ac82-faaa2d42a27c None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:18:56.285592 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-e7a4aceb-2193-459d-ac82-faaa2d42a27c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:18:56.329093 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:18:56.442118 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 719/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 21:18:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:18:56.484661 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28e01c3-4fda-4aea-ae15-7d0f73ea11dd None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:18:56.486439 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-a28e01c3-4fda-4aea-ae15-7d0f73ea11dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:18:56.636751 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28e01c3-4fda-4aea-ae15-7d0f73ea11dd None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:18:56.668747 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:18:56.725032 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301f5468-f022-487c-af2a-d38b280fea72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:18:56.725705 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f5468-f022-487c-af2a-d38b280fea72 service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:18:56.726031 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f5468-f022-487c-af2a-d38b280fea72 service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:18:56.726349 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f5468-f022-487c-af2a-d38b280fea72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:18:56.729119 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-301f5468-f022-487c-af2a-d38b280fea72 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:18:56.855423 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301f5468-f022-487c-af2a-d38b280fea72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:18:56.856904 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301f5468-f022-487c-af2a-d38b280fea72 service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:18:56.877662 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 739/1458] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:18:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:18:59.290177 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583651d8-0d58-4fd6-b335-e472fa78b472 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:18:59.291295 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583651d8-0d58-4fd6-b335-e472fa78b472 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:18:59.291295 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583651d8-0d58-4fd6-b335-e472fa78b472 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:18:59.293873 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 720/1459] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 21:18:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:18:59.301343 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94a1769-b17f-40f5-aead-02b6aa60ccab None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:18:59.301871 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94a1769-b17f-40f5-aead-02b6aa60ccab None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:18:59.307172 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94a1769-b17f-40f5-aead-02b6aa60ccab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:18:59.378324 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-e94a1769-b17f-40f5-aead-02b6aa60ccab None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:18:59.379593 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-e94a1769-b17f-40f5-aead-02b6aa60ccab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:18:59.389040 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:18:59.515238 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 740/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 21:18:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:18:59.560450 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583651d8-0d58-4fd6-b335-e472fa78b472 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:18:59.563281 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-583651d8-0d58-4fd6-b335-e472fa78b472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:18:59.717135 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583651d8-0d58-4fd6-b335-e472fa78b472 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:18:59.799219 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:18:59.821812 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dfdcec-f652-4cf6-9f35-d36d162da0a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:18:59.822444 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dfdcec-f652-4cf6-9f35-d36d162da0a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:18:59.822723 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dfdcec-f652-4cf6-9f35-d36d162da0a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:18:59.823082 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dfdcec-f652-4cf6-9f35-d36d162da0a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:18:59.824751 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-a4dfdcec-f652-4cf6-9f35-d36d162da0a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:18:59.981502 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dfdcec-f652-4cf6-9f35-d36d162da0a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:18:59.983057 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dfdcec-f652-4cf6-9f35-d36d162da0a2 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:19:00.006979 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 721/1461] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:18:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:19:02.207939 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77c8ce7-a935-4e13-a815-38756eb6e195 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:19:02.208500 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77c8ce7-a935-4e13-a815-38756eb6e195 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:19:02.208500 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77c8ce7-a935-4e13-a815-38756eb6e195 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:19:02.210013 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 741/1462] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 21:19:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:19:02.216473 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c6bf84-a3af-48cf-a75a-57024eb87cb6 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:19:02.216971 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c6bf84-a3af-48cf-a75a-57024eb87cb6 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:19:02.217066 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c6bf84-a3af-48cf-a75a-57024eb87cb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:19:02.289381 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-b3c6bf84-a3af-48cf-a75a-57024eb87cb6 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:19:02.290643 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-b3c6bf84-a3af-48cf-a75a-57024eb87cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:19:02.339469 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:19:02.402690 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 722/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 21:19:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:19:02.444821 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77c8ce7-a935-4e13-a815-38756eb6e195 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:19:02.447270 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-e77c8ce7-a935-4e13-a815-38756eb6e195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:19:02.559418 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77c8ce7-a935-4e13-a815-38756eb6e195 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:19:02.601959 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:19:02.642671 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b342e1-7e92-4314-8d5c-86ec03c5695b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:19:02.643285 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b342e1-7e92-4314-8d5c-86ec03c5695b service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:19:02.643586 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b342e1-7e92-4314-8d5c-86ec03c5695b service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:19:02.643859 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b342e1-7e92-4314-8d5c-86ec03c5695b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:19:02.645805 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-47b342e1-7e92-4314-8d5c-86ec03c5695b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:19:02.765518 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b342e1-7e92-4314-8d5c-86ec03c5695b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:19:02.770845 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b342e1-7e92-4314-8d5c-86ec03c5695b service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:19:02.786816 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 742/1464] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:19:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:19:05.042494 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dce31ce-09b7-409d-9464-ec996dd765da None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:19:05.043070 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dce31ce-09b7-409d-9464-ec996dd765da None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:19:05.043680 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dce31ce-09b7-409d-9464-ec996dd765da None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:19:05.046150 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 723/1465] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 21:19:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:19:05.053397 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f319202-d03d-455f-9a54-5a86a7d1865b None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:19:05.054047 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f319202-d03d-455f-9a54-5a86a7d1865b None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:19:05.054047 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f319202-d03d-455f-9a54-5a86a7d1865b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:19:05.141969 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-0f319202-d03d-455f-9a54-5a86a7d1865b None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:19:05.143380 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-0f319202-d03d-455f-9a54-5a86a7d1865b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:19:05.182104 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:19:05.283538 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 743/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 21:19:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:19:05.339290 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dce31ce-09b7-409d-9464-ec996dd765da None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:19:05.341387 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-2dce31ce-09b7-409d-9464-ec996dd765da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:19:05.492982 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dce31ce-09b7-409d-9464-ec996dd765da None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:19:05.594793 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:19:05.607035 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc61d874-7fa6-4686-84fa-f97696b5632d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:19:05.619284 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc61d874-7fa6-4686-84fa-f97696b5632d service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:19:05.619284 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc61d874-7fa6-4686-84fa-f97696b5632d service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:19:05.619284 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc61d874-7fa6-4686-84fa-f97696b5632d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:19:05.619284 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-bc61d874-7fa6-4686-84fa-f97696b5632d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:19:05.743894 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc61d874-7fa6-4686-84fa-f97696b5632d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:19:05.745343 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc61d874-7fa6-4686-84fa-f97696b5632d service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:19:05.765104 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 724/1467] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:19:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:19:08.070807 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b74eb16-6c9a-44f3-8e5c-4bcaab7ad351 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:19:08.071446 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b74eb16-6c9a-44f3-8e5c-4bcaab7ad351 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:19:08.071635 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b74eb16-6c9a-44f3-8e5c-4bcaab7ad351 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:19:08.073211 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 744/1468] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 21:19:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:19:08.079662 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd87831a-ae54-4453-9daf-6bfd3b34237e None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:19:08.080410 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd87831a-ae54-4453-9daf-6bfd3b34237e None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:19:08.080410 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd87831a-ae54-4453-9daf-6bfd3b34237e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:19:08.157408 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-fd87831a-ae54-4453-9daf-6bfd3b34237e None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:19:08.158840 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-fd87831a-ae54-4453-9daf-6bfd3b34237e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:19:08.235718 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:19:08.315723 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 725/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 21:19:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:19:08.357576 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b74eb16-6c9a-44f3-8e5c-4bcaab7ad351 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:19:08.359237 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-9b74eb16-6c9a-44f3-8e5c-4bcaab7ad351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:19:08.496158 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b74eb16-6c9a-44f3-8e5c-4bcaab7ad351 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:19:08.558803 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:19:08.594281 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3652bf-be01-45ce-89c6-26c490438bc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:19:08.594912 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3652bf-be01-45ce-89c6-26c490438bc6 service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:19:08.595193 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3652bf-be01-45ce-89c6-26c490438bc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:19:08.595498 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3652bf-be01-45ce-89c6-26c490438bc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:19:08.597207 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-0b3652bf-be01-45ce-89c6-26c490438bc6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:19:08.726077 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3652bf-be01-45ce-89c6-26c490438bc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:19:08.732114 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3652bf-be01-45ce-89c6-26c490438bc6 service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:19:08.757706 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 745/1470] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:19:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:19:18.312149 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=22 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:19:18.755087 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=66 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:19:29.846030 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f608b18-f1e2-4c94-92dc-65fe2ef847d8 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:19:29.846641 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f608b18-f1e2-4c94-92dc-65fe2ef847d8 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:19:29.846641 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f608b18-f1e2-4c94-92dc-65fe2ef847d8 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:19:29.849178 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 726/1471] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:19:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:19:29.854280 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6f7e14-2aaf-48c9-8c37-f31af207fe75 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:19:29.854826 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6f7e14-2aaf-48c9-8c37-f31af207fe75 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:19:29.854826 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6f7e14-2aaf-48c9-8c37-f31af207fe75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:19:29.955963 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-3d6f7e14-2aaf-48c9-8c37-f31af207fe75 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:19:29.957225 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-3d6f7e14-2aaf-48c9-8c37-f31af207fe75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:19:30.136459 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 746/1472] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:19:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:19:30.247036 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf1be6a-f6fb-4cf7-bcde-f83e8cf7ef40 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:19:30.247451 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf1be6a-f6fb-4cf7-bcde-f83e8cf7ef40 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:19:30.249643 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf1be6a-f6fb-4cf7-bcde-f83e8cf7ef40 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:19:30.249643 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 727/1473] 10.222.0.25 () {54 vars in 966 bytes} [Thu Aug 4 21:19:30 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:19:30.256947 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f89e7f-7650-4cbf-9346-436fd2212ec2 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:19:30.257311 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f89e7f-7650-4cbf-9346-436fd2212ec2 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:19:30.257769 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f89e7f-7650-4cbf-9346-436fd2212ec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:19:30.368871 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-88f89e7f-7650-4cbf-9346-436fd2212ec2 None None] MFA Rules not processed for user `f6885c7d929f4f8e96d3890724c3b53f`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:19:30.370757 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-88f89e7f-7650-4cbf-9346-436fd2212ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:19:30.397485 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:19:30.474144 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 747/1474] 10.222.0.25 () {58 vars in 1063 bytes} [Thu Aug 4 21:19:30 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:19:30.482165 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf1be6a-f6fb-4cf7-bcde-f83e8cf7ef40 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:19:30.483757 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-bbf1be6a-f6fb-4cf7-bcde-f83e8cf7ef40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:19:30.606693 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf1be6a-f6fb-4cf7-bcde-f83e8cf7ef40 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:19:30.703368 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26b6715-410d-4e15-af06-43a5966a5483 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6885c7d929f4f8e96d3890724c3b53f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:19:30.704005 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26b6715-410d-4e15-af06-43a5966a5483 service glance] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:19:30.704279 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26b6715-410d-4e15-af06-43a5966a5483 service glance] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:19:30.704546 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26b6715-410d-4e15-af06-43a5966a5483 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:19:30.706330 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-a26b6715-410d-4e15-af06-43a5966a5483 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:19:30.770399 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:19:30.832757 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26b6715-410d-4e15-af06-43a5966a5483 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:19:30.834226 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26b6715-410d-4e15-af06-43a5966a5483 service glance] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:19:30.854712 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 728/1475] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:19:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:19:33.232390 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e804e207-1520-4af6-8a8e-9d9ab06b72cb None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:19:33.235802 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e804e207-1520-4af6-8a8e-9d9ab06b72cb None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:19:33.235802 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e804e207-1520-4af6-8a8e-9d9ab06b72cb None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:19:33.235802 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 748/1476] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:19:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 21:19:33.240745 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac93060-555c-4dae-9c34-50a38f68a1ef None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:19:33.240745 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac93060-555c-4dae-9c34-50a38f68a1ef None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:19:33.241372 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac93060-555c-4dae-9c34-50a38f68a1ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:19:33.313882 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-eac93060-555c-4dae-9c34-50a38f68a1ef None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:19:33.315221 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-eac93060-555c-4dae-9c34-50a38f68a1ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:19:33.442621 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 729/1477] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:19:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:19:33.468736 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e804e207-1520-4af6-8a8e-9d9ab06b72cb None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:19:33.471717 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-e804e207-1520-4af6-8a8e-9d9ab06b72cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:19:33.678528 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e804e207-1520-4af6-8a8e-9d9ab06b72cb None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:19:33.734112 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:19:33.784430 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0123c6fb-f8ab-4161-a37d-8ed0a8ca1626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:19:33.785119 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0123c6fb-f8ab-4161-a37d-8ed0a8ca1626 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:19:33.785811 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0123c6fb-f8ab-4161-a37d-8ed0a8ca1626 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:19:33.786274 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0123c6fb-f8ab-4161-a37d-8ed0a8ca1626 admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:19:33.787245 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0123c6fb-f8ab-4161-a37d-8ed0a8ca1626 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:19:33.789160 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: WARNING py.warnings [None req-0123c6fb-f8ab-4161-a37d-8ed0a8ca1626 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 21:19:33.789160 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: warnings.warn(msg) Aug 04 21:19:33.789160 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: Aug 04 21:19:33.790326 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0123c6fb-f8ab-4161-a37d-8ed0a8ca1626 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:19:33.812650 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 749/1478] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Aug 4 21:19:33 2022] GET /identity/v3/projects => generated 2553 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:19:35.944275 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116f0a98-22ad-43d8-9f96-10e29af9d283 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:19:35.944892 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116f0a98-22ad-43d8-9f96-10e29af9d283 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:19:35.944892 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116f0a98-22ad-43d8-9f96-10e29af9d283 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:19:35.946272 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 730/1479] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:19:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:19:35.952953 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6573eb7-dba5-4df5-afa7-410a0d854c9f None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:19:35.953502 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6573eb7-dba5-4df5-afa7-410a0d854c9f None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:19:35.953905 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6573eb7-dba5-4df5-afa7-410a0d854c9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:19:36.032988 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-c6573eb7-dba5-4df5-afa7-410a0d854c9f None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:19:36.034475 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-c6573eb7-dba5-4df5-afa7-410a0d854c9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:19:36.201170 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 750/1480] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:19:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:19:36.247385 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116f0a98-22ad-43d8-9f96-10e29af9d283 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:19:36.248995 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-116f0a98-22ad-43d8-9f96-10e29af9d283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:19:36.259076 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:19:36.420330 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116f0a98-22ad-43d8-9f96-10e29af9d283 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:19:36.510130 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:19:36.550720 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79506658-6f0e-4c73-9e5b-5d4041305628 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a2ce1df87fa47e5be301916bf3245fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:19:36.551363 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79506658-6f0e-4c73-9e5b-5d4041305628 service nova] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:19:36.551647 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79506658-6f0e-4c73-9e5b-5d4041305628 service nova] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:19:36.551918 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79506658-6f0e-4c73-9e5b-5d4041305628 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:19:36.553773 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-79506658-6f0e-4c73-9e5b-5d4041305628 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:19:36.666877 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79506658-6f0e-4c73-9e5b-5d4041305628 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:19:36.669215 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79506658-6f0e-4c73-9e5b-5d4041305628 service nova] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:19:36.688518 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 731/1481] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:19:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:19:39.017215 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9be22d-0f1c-4787-adf5-ef4dcfea8b5c None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:19:39.017785 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9be22d-0f1c-4787-adf5-ef4dcfea8b5c None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:19:39.017785 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9be22d-0f1c-4787-adf5-ef4dcfea8b5c None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:19:39.019127 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 751/1482] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:19:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:19:39.024903 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262c8f8b-379f-411a-bd49-d3083d5e0be3 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:19:39.025395 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262c8f8b-379f-411a-bd49-d3083d5e0be3 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:19:39.025520 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262c8f8b-379f-411a-bd49-d3083d5e0be3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:19:39.101754 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-262c8f8b-379f-411a-bd49-d3083d5e0be3 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:19:39.102913 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-262c8f8b-379f-411a-bd49-d3083d5e0be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:19:39.154356 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:19:39.210398 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 732/1483] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:19:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:19:39.248573 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9be22d-0f1c-4787-adf5-ef4dcfea8b5c None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:19:39.250131 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-bc9be22d-0f1c-4787-adf5-ef4dcfea8b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:19:39.276842 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:19:39.373672 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9be22d-0f1c-4787-adf5-ef4dcfea8b5c None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:19:39.459383 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297638d6-73f4-47c3-b27d-7d93437cc168 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:19:39.460008 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297638d6-73f4-47c3-b27d-7d93437cc168 service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:19:39.460355 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297638d6-73f4-47c3-b27d-7d93437cc168 service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:19:39.461374 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297638d6-73f4-47c3-b27d-7d93437cc168 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:19:39.463855 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-297638d6-73f4-47c3-b27d-7d93437cc168 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:19:39.568605 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:19:39.606662 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297638d6-73f4-47c3-b27d-7d93437cc168 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:19:39.608186 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297638d6-73f4-47c3-b27d-7d93437cc168 service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:19:39.629728 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 752/1484] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:19:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:19:41.825988 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6856023-5ebd-417e-89d3-24a9e8650a24 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:19:41.826920 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6856023-5ebd-417e-89d3-24a9e8650a24 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:19:41.826920 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6856023-5ebd-417e-89d3-24a9e8650a24 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:19:41.829530 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 733/1485] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:19:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:19:41.838788 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ba00d7-47ba-4fc9-a2a1-ad99ac517d8e None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:19:41.839479 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ba00d7-47ba-4fc9-a2a1-ad99ac517d8e None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:19:41.839479 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ba00d7-47ba-4fc9-a2a1-ad99ac517d8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:19:41.911781 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-88ba00d7-47ba-4fc9-a2a1-ad99ac517d8e None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:19:41.913015 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-88ba00d7-47ba-4fc9-a2a1-ad99ac517d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:19:42.018415 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 753/1486] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:19:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:19:42.061240 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6856023-5ebd-417e-89d3-24a9e8650a24 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:19:42.062849 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-a6856023-5ebd-417e-89d3-24a9e8650a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:19:42.175914 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6856023-5ebd-417e-89d3-24a9e8650a24 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:19:42.263609 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:19:42.265446 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dc0b07-57a1-407d-bee4-b074111d8ef4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:19:42.266107 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dc0b07-57a1-407d-bee4-b074111d8ef4 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:19:42.266439 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dc0b07-57a1-407d-bee4-b074111d8ef4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:19:42.266716 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dc0b07-57a1-407d-bee4-b074111d8ef4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:19:42.268820 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-54dc0b07-57a1-407d-bee4-b074111d8ef4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:19:42.393696 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dc0b07-57a1-407d-bee4-b074111d8ef4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:19:42.395212 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dc0b07-57a1-407d-bee4-b074111d8ef4 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:19:42.416608 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 734/1487] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:19:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:19:44.765451 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796dbac2-9a60-448e-a586-76e304117fc9 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:19:44.766135 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796dbac2-9a60-448e-a586-76e304117fc9 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:19:44.766361 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796dbac2-9a60-448e-a586-76e304117fc9 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:19:44.768249 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 754/1488] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:19:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:19:44.775404 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799f7897-f0be-41de-9be1-bfe57dbd711f None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:19:44.775930 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799f7897-f0be-41de-9be1-bfe57dbd711f None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:19:44.776053 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799f7897-f0be-41de-9be1-bfe57dbd711f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:19:44.856159 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-799f7897-f0be-41de-9be1-bfe57dbd711f None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:19:44.857491 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-799f7897-f0be-41de-9be1-bfe57dbd711f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:19:44.920796 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:19:44.976412 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 735/1489] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:19:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:19:45.022427 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796dbac2-9a60-448e-a586-76e304117fc9 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:19:45.024535 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-796dbac2-9a60-448e-a586-76e304117fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:19:45.074689 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:19:45.161901 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796dbac2-9a60-448e-a586-76e304117fc9 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:19:45.250662 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3979105-0721-4943-98fb-811263f79f3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:19:45.251294 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3979105-0721-4943-98fb-811263f79f3b service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:19:45.251611 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3979105-0721-4943-98fb-811263f79f3b service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:19:45.251890 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3979105-0721-4943-98fb-811263f79f3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:19:45.253940 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-c3979105-0721-4943-98fb-811263f79f3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:19:45.350596 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:19:45.387820 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3979105-0721-4943-98fb-811263f79f3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:19:45.389286 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3979105-0721-4943-98fb-811263f79f3b service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:19:45.411126 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 755/1490] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:19:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:19:48.196434 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb2f4f5-6898-4826-947e-d3fb3168239a None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:19:48.197001 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb2f4f5-6898-4826-947e-d3fb3168239a None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:19:48.197001 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb2f4f5-6898-4826-947e-d3fb3168239a None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:19:48.198354 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 736/1491] 10.222.0.25 () {54 vars in 927 bytes} [Thu Aug 4 21:19:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:19:48.204595 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5c11ee-30a9-4006-8fc3-f8abd5cc4049 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:19:48.205149 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5c11ee-30a9-4006-8fc3-f8abd5cc4049 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:19:48.205149 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5c11ee-30a9-4006-8fc3-f8abd5cc4049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:19:48.279175 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-1d5c11ee-30a9-4006-8fc3-f8abd5cc4049 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:19:48.280421 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-1d5c11ee-30a9-4006-8fc3-f8abd5cc4049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:19:48.396361 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 756/1492] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Aug 4 21:19:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:19:48.437662 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb2f4f5-6898-4826-947e-d3fb3168239a None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:19:48.439443 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-fbb2f4f5-6898-4826-947e-d3fb3168239a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:19:48.565627 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb2f4f5-6898-4826-947e-d3fb3168239a None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:19:48.656237 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f923b67d-661d-4577-86fb-185ee76ac718 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:19:48.657179 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f923b67d-661d-4577-86fb-185ee76ac718 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:19:48.657305 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f923b67d-661d-4577-86fb-185ee76ac718 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:19:48.657616 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f923b67d-661d-4577-86fb-185ee76ac718 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:19:48.659379 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-f923b67d-661d-4577-86fb-185ee76ac718 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:19:48.670663 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:19:48.835355 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f923b67d-661d-4577-86fb-185ee76ac718 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:19:48.837228 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f923b67d-661d-4577-86fb-185ee76ac718 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:19:48.858598 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 737/1493] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:19:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:19:58.394497 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=82 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:19:58.856104 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=50 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:23.533671 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8016ad6-08be-4bb7-8cca-5e2c3364dcf3 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:23.543166 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8016ad6-08be-4bb7-8cca-5e2c3364dcf3 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:23.543166 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8016ad6-08be-4bb7-8cca-5e2c3364dcf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:23.647968 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-d8016ad6-08be-4bb7-8cca-5e2c3364dcf3 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:21:23.649218 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-d8016ad6-08be-4bb7-8cca-5e2c3364dcf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:21:23.793809 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 757/1494] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:21:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:21:23.802495 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155eb789-6d6f-43de-aa9a-040d1f1aef8a None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:23.803367 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155eb789-6d6f-43de-aa9a-040d1f1aef8a None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:23.803367 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155eb789-6d6f-43de-aa9a-040d1f1aef8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:23.829668 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-155eb789-6d6f-43de-aa9a-040d1f1aef8a None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:21:23.831045 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-155eb789-6d6f-43de-aa9a-040d1f1aef8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:21:23.841354 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 738/1495] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:21:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:21:23.846288 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8016ad6-08be-4bb7-8cca-5e2c3364dcf3 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:23.857047 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8016ad6-08be-4bb7-8cca-5e2c3364dcf3 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:23.862167 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fb9a74-efaa-41fa-8659-d3a4e25f5cf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:23.862801 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fb9a74-efaa-41fa-8659-d3a4e25f5cf3 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:23.863081 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fb9a74-efaa-41fa-8659-d3a4e25f5cf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:23.863383 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fb9a74-efaa-41fa-8659-d3a4e25f5cf3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:23.864097 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fb9a74-efaa-41fa-8659-d3a4e25f5cf3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:23.864397 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fb9a74-efaa-41fa-8659-d3a4e25f5cf3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:23.866108 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: WARNING py.warnings [None req-15fb9a74-efaa-41fa-8659-d3a4e25f5cf3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 04 21:21:23.866108 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: warnings.warn(msg) Aug 04 21:21:23.866108 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: Aug 04 21:21:23.866531 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fb9a74-efaa-41fa-8659-d3a4e25f5cf3 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:23.880166 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 758/1496] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Aug 4 21:21:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:23.889597 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155eb789-6d6f-43de-aa9a-040d1f1aef8a None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:23.899588 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155eb789-6d6f-43de-aa9a-040d1f1aef8a None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:23.905326 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf06c61f-e48a-40d4-b810-af33f2944377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:23.906297 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf06c61f-e48a-40d4-b810-af33f2944377 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:23.906297 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf06c61f-e48a-40d4-b810-af33f2944377 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:23.906541 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf06c61f-e48a-40d4-b810-af33f2944377 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:23.907244 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf06c61f-e48a-40d4-b810-af33f2944377 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:23.909085 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: WARNING py.warnings [None req-cf06c61f-e48a-40d4-b810-af33f2944377 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 21:21:23.909085 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: warnings.warn(msg) Aug 04 21:21:23.909085 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: Aug 04 21:21:23.910332 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf06c61f-e48a-40d4-b810-af33f2944377 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:24.051920 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 739/1497] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:21:23 2022] POST /identity/v3/projects => generated 379 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:21:24.056649 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fb9a74-efaa-41fa-8659-d3a4e25f5cf3 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:24.068245 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fb9a74-efaa-41fa-8659-d3a4e25f5cf3 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:24.133659 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:24.143948 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a953247c-86d5-4d0d-9ce3-0f19ba8e5b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:24.144592 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a953247c-86d5-4d0d-9ce3-0f19ba8e5b42 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:24.144868 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a953247c-86d5-4d0d-9ce3-0f19ba8e5b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:24.145143 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a953247c-86d5-4d0d-9ce3-0f19ba8e5b42 admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:24.146076 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a953247c-86d5-4d0d-9ce3-0f19ba8e5b42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:24.147125 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: WARNING py.warnings [None req-a953247c-86d5-4d0d-9ce3-0f19ba8e5b42 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 21:21:24.147125 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: warnings.warn(msg) Aug 04 21:21:24.147125 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: Aug 04 21:21:24.148380 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a953247c-86d5-4d0d-9ce3-0f19ba8e5b42 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:24.259670 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 759/1498] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:21:24 2022] POST /identity/v3/users => generated 290 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:21:24.263767 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf06c61f-e48a-40d4-b810-af33f2944377 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:24.274889 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf06c61f-e48a-40d4-b810-af33f2944377 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:24.281181 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5108ef5e-9b8b-4a07-8f8f-391e5b319c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:24.282236 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5108ef5e-9b8b-4a07-8f8f-391e5b319c63 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:24.283095 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5108ef5e-9b8b-4a07-8f8f-391e5b319c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:24.283519 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5108ef5e-9b8b-4a07-8f8f-391e5b319c63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:24.284486 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5108ef5e-9b8b-4a07-8f8f-391e5b319c63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:24.285974 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: WARNING py.warnings [None req-5108ef5e-9b8b-4a07-8f8f-391e5b319c63 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 04 21:21:24.285974 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: warnings.warn(msg) Aug 04 21:21:24.285974 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: Aug 04 21:21:24.287294 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5108ef5e-9b8b-4a07-8f8f-391e5b319c63 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:24.306876 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 740/1499] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:21:24 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:21:24.311751 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a953247c-86d5-4d0d-9ce3-0f19ba8e5b42 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:24.322696 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a953247c-86d5-4d0d-9ce3-0f19ba8e5b42 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:24.328454 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4530fc30-abb3-4a7b-a45e-0537aafe57b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:24.329102 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4530fc30-abb3-4a7b-a45e-0537aafe57b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:24.329396 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4530fc30-abb3-4a7b-a45e-0537aafe57b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:24.329667 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4530fc30-abb3-4a7b-a45e-0537aafe57b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:24.330374 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4530fc30-abb3-4a7b-a45e-0537aafe57b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:24.331503 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: WARNING py.warnings [None req-4530fc30-abb3-4a7b-a45e-0537aafe57b2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 04 21:21:24.331503 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: warnings.warn(msg) Aug 04 21:21:24.331503 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: Aug 04 21:21:24.338858 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4530fc30-abb3-4a7b-a45e-0537aafe57b2 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:24.348561 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 760/1500] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:21:24 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:21:24.353124 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5108ef5e-9b8b-4a07-8f8f-391e5b319c63 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:24.366948 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5108ef5e-9b8b-4a07-8f8f-391e5b319c63 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:24.372792 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1905cfd7-4e3e-4bbf-8358-ca46917a5b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:24.373496 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1905cfd7-4e3e-4bbf-8358-ca46917a5b1a admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:24.373774 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1905cfd7-4e3e-4bbf-8358-ca46917a5b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:24.374049 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1905cfd7-4e3e-4bbf-8358-ca46917a5b1a admin admin] PATH_INFO: `/v3/projects/8176c40167314603a4c80e78ffa7b25d/users/7cc680a33c764b76995a75feecf4dbad/roles/8feebc60363c4feb870718b1721e5324` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:24.394520 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1905cfd7-4e3e-4bbf-8358-ca46917a5b1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8176c40167314603a4c80e78ffa7b25d, user_id=7cc680a33c764b76995a75feecf4dbad, role_id=8feebc60363c4feb870718b1721e5324)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:24.395629 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: WARNING py.warnings [None req-1905cfd7-4e3e-4bbf-8358-ca46917a5b1a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 21:21:24.395629 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: warnings.warn(msg) Aug 04 21:21:24.395629 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: Aug 04 21:21:24.397753 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1905cfd7-4e3e-4bbf-8358-ca46917a5b1a admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:24.479173 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 741/1501] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:21:24 2022] PUT /identity/v3/projects/8176c40167314603a4c80e78ffa7b25d/users/7cc680a33c764b76995a75feecf4dbad/roles/8feebc60363c4feb870718b1721e5324 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:21:24.492007 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c20ed5-a9b4-46a7-82fe-b14e79c23fe0 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:24.492294 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c20ed5-a9b4-46a7-82fe-b14e79c23fe0 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:24.492571 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c20ed5-a9b4-46a7-82fe-b14e79c23fe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:24.529701 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-62c20ed5-a9b4-46a7-82fe-b14e79c23fe0 None None] MFA Rules not processed for user `7cc680a33c764b76995a75feecf4dbad`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:21:24.531591 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-62c20ed5-a9b4-46a7-82fe-b14e79c23fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:21:24.624338 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 761/1502] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:21:24 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:21:24.639905 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e707e6fb-35d9-4833-a40d-15ebb698b9e6 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:24.640194 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e707e6fb-35d9-4833-a40d-15ebb698b9e6 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:24.640476 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e707e6fb-35d9-4833-a40d-15ebb698b9e6 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:24.641898 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 742/1503] 10.222.0.25 () {54 vars in 964 bytes} [Thu Aug 4 21:21:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 21:21:24.656722 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c905af45-1a16-4e37-838d-84a1b463f295 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:24.657004 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c905af45-1a16-4e37-838d-84a1b463f295 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:24.657277 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c905af45-1a16-4e37-838d-84a1b463f295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:24.723595 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-c905af45-1a16-4e37-838d-84a1b463f295 None None] MFA Rules not processed for user `e31f2728056b460fa474478eb85ca1d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:21:24.724930 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-c905af45-1a16-4e37-838d-84a1b463f295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:21:24.815291 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 762/1504] 10.222.0.25 () {58 vars in 1061 bytes} [Thu Aug 4 21:21:24 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:21:24.829305 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e707e6fb-35d9-4833-a40d-15ebb698b9e6 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:24.842352 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e707e6fb-35d9-4833-a40d-15ebb698b9e6 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:24.842352 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02c2ff6-4275-43e8-a0f4-ecd649208a82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e31f2728056b460fa474478eb85ca1d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:24.847113 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c2ff6-4275-43e8-a0f4-ecd649208a82 service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:24.847423 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c2ff6-4275-43e8-a0f4-ecd649208a82 service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:24.847707 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c2ff6-4275-43e8-a0f4-ecd649208a82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:24.857175 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02c2ff6-4275-43e8-a0f4-ecd649208a82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:24.858639 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02c2ff6-4275-43e8-a0f4-ecd649208a82 service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:24.869667 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 743/1505] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:21:24 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:21:24.933777 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfd897b-ee1d-4804-996d-1b4ebeb3c666 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:24.934075 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfd897b-ee1d-4804-996d-1b4ebeb3c666 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:24.934375 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfd897b-ee1d-4804-996d-1b4ebeb3c666 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:24.935709 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 763/1506] 10.222.0.25 () {52 vars in 830 bytes} [Thu Aug 4 21:21:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:21:24.940202 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02c2ff6-4275-43e8-a0f4-ecd649208a82 service cinder] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:25.013096 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02c2ff6-4275-43e8-a0f4-ecd649208a82 service cinder] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:25.018378 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4219c0-dd0e-44c5-a4c5-e66bb10d40e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:25.019047 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4219c0-dd0e-44c5-a4c5-e66bb10d40e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:25.019355 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4219c0-dd0e-44c5-a4c5-e66bb10d40e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:25.019793 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4219c0-dd0e-44c5-a4c5-e66bb10d40e4 admin admin] PATH_INFO: `/v3/users/7cc680a33c764b76995a75feecf4dbad` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:25.020863 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4219c0-dd0e-44c5-a4c5-e66bb10d40e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cc680a33c764b76995a75feecf4dbad)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:25.021909 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: WARNING py.warnings [None req-4f4219c0-dd0e-44c5-a4c5-e66bb10d40e4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 21:21:25.021909 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: warnings.warn(msg) Aug 04 21:21:25.021909 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: Aug 04 21:21:25.023140 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4219c0-dd0e-44c5-a4c5-e66bb10d40e4 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:25.255209 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-4f4219c0-dd0e-44c5-a4c5-e66bb10d40e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cc680a33c764b76995a75feecf4dbad'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:21:25.294253 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-4f4219c0-dd0e-44c5-a4c5-e66bb10d40e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cc680a33c764b76995a75feecf4dbad'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:21:25.296645 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-4f4219c0-dd0e-44c5-a4c5-e66bb10d40e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cc680a33c764b76995a75feecf4dbad'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:21:25.316672 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-4f4219c0-dd0e-44c5-a4c5-e66bb10d40e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cc680a33c764b76995a75feecf4dbad'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:21:25.317388 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-4f4219c0-dd0e-44c5-a4c5-e66bb10d40e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cc680a33c764b76995a75feecf4dbad'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:21:25.321513 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:25.441321 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 744/1507] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:21:24 2022] DELETE /identity/v3/users/7cc680a33c764b76995a75feecf4dbad => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:21:25.446991 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4f2d62-afff-40a7-aab1-259223116515 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:25.447827 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4f2d62-afff-40a7-aab1-259223116515 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:25.448108 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4f2d62-afff-40a7-aab1-259223116515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:25.470378 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-7f4f2d62-afff-40a7-aab1-259223116515 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:21:25.471721 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-7f4f2d62-afff-40a7-aab1-259223116515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:21:25.489798 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:25.546197 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 764/1508] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:21:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:21:25.556528 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4219c0-dd0e-44c5-a4c5-e66bb10d40e4 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:25.558936 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-4f4219c0-dd0e-44c5-a4c5-e66bb10d40e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:21:25.668264 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4219c0-dd0e-44c5-a4c5-e66bb10d40e4 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:25.673901 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59f4799-405a-4e1b-818e-331041190ed0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:25.674838 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59f4799-405a-4e1b-818e-331041190ed0 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:25.675272 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59f4799-405a-4e1b-818e-331041190ed0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:25.675730 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59f4799-405a-4e1b-818e-331041190ed0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:25.691621 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59f4799-405a-4e1b-818e-331041190ed0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:25.694008 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59f4799-405a-4e1b-818e-331041190ed0 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:25.708367 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 745/1509] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:21:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:21:26.319374 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4f2d62-afff-40a7-aab1-259223116515 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:26.321051 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-7f4f2d62-afff-40a7-aab1-259223116515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:21:26.341001 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4f2d62-afff-40a7-aab1-259223116515 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:26.350984 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ae9041-8f45-4d94-82b5-531e55f2bccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:26.352141 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ae9041-8f45-4d94-82b5-531e55f2bccc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:26.352630 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ae9041-8f45-4d94-82b5-531e55f2bccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:26.353082 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ae9041-8f45-4d94-82b5-531e55f2bccc admin admin] PATH_INFO: `/v3/projects/8176c40167314603a4c80e78ffa7b25d` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:26.355702 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ae9041-8f45-4d94-82b5-531e55f2bccc admin admin] RBAC: Authorizing `identity:delete_project(project_id=8176c40167314603a4c80e78ffa7b25d)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:26.357497 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: WARNING py.warnings [None req-e3ae9041-8f45-4d94-82b5-531e55f2bccc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 21:21:26.357497 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: warnings.warn(msg) Aug 04 21:21:26.357497 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: Aug 04 21:21:26.359810 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ae9041-8f45-4d94-82b5-531e55f2bccc admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:26.553163 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-e3ae9041-8f45-4d94-82b5-531e55f2bccc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8176c40167314603a4c80e78ffa7b25d', 'request_id': 'req-e3ae9041-8f45-4d94-82b5-531e55f2bccc'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:21:26.561346 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-e3ae9041-8f45-4d94-82b5-531e55f2bccc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8176c40167314603a4c80e78ffa7b25d', 'request_id': 'req-e3ae9041-8f45-4d94-82b5-531e55f2bccc'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:21:26.729241 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-e3ae9041-8f45-4d94-82b5-531e55f2bccc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8176c40167314603a4c80e78ffa7b25d', 'request_id': 'req-e3ae9041-8f45-4d94-82b5-531e55f2bccc'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:21:26.749652 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-e3ae9041-8f45-4d94-82b5-531e55f2bccc admin admin] The token cache is being invalidate because project 8176c40167314603a4c80e78ffa7b25d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:21:26.749944 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-e3ae9041-8f45-4d94-82b5-531e55f2bccc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:21:26.750786 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-e3ae9041-8f45-4d94-82b5-531e55f2bccc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:21:26.752438 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 765/1510] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:21:26 2022] DELETE /identity/v3/projects/8176c40167314603a4c80e78ffa7b25d => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:21:35.703033 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:DELETE=1,keystone:SELECT=37 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:36.746508 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:03.685354 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ae8cd-74f2-4710-8764-1dfba689e58d None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:03.685959 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ae8cd-74f2-4710-8764-1dfba689e58d None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:03.685959 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ae8cd-74f2-4710-8764-1dfba689e58d None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:03.687363 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 746/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 21:23:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:23:03.693329 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb655808-3c7b-4fa7-a24a-3c618a53cf3e None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:03.694026 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb655808-3c7b-4fa7-a24a-3c618a53cf3e None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:03.694026 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb655808-3c7b-4fa7-a24a-3c618a53cf3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:03.716742 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-fb655808-3c7b-4fa7-a24a-3c618a53cf3e None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:23:03.717971 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-fb655808-3c7b-4fa7-a24a-3c618a53cf3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:23:03.795464 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 766/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 21:23:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:23:03.951892 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9ae8cd-74f2-4710-8764-1dfba689e58d None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:03.954253 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-8b9ae8cd-74f2-4710-8764-1dfba689e58d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:23:04.082182 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9ae8cd-74f2-4710-8764-1dfba689e58d None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:04.091243 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690aba6f-fc85-43b6-9543-1ffbe0e49582 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a2ce1df87fa47e5be301916bf3245fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:04.092316 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690aba6f-fc85-43b6-9543-1ffbe0e49582 service nova] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:04.092816 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690aba6f-fc85-43b6-9543-1ffbe0e49582 service nova] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:04.093349 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690aba6f-fc85-43b6-9543-1ffbe0e49582 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:04.103162 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690aba6f-fc85-43b6-9543-1ffbe0e49582 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:04.106325 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690aba6f-fc85-43b6-9543-1ffbe0e49582 service nova] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:04.129506 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 747/1513] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:23:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:23:04.987886 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7802ff1-03a7-4a54-995f-c00c210c4824 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:04.988438 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7802ff1-03a7-4a54-995f-c00c210c4824 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:04.988747 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7802ff1-03a7-4a54-995f-c00c210c4824 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:04.990237 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 767/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 21:23:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:23:04.995804 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2370bea2-43d6-4300-9438-4654d09e1abe None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:04.996349 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2370bea2-43d6-4300-9438-4654d09e1abe None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:04.996349 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2370bea2-43d6-4300-9438-4654d09e1abe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:05.019509 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-2370bea2-43d6-4300-9438-4654d09e1abe None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:23:05.020656 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-2370bea2-43d6-4300-9438-4654d09e1abe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:23:05.028807 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 748/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 21:23:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:23:05.174068 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f0208d-acfd-4c06-9563-3a8603aa264c None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:05.174641 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f0208d-acfd-4c06-9563-3a8603aa264c None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:05.175022 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f0208d-acfd-4c06-9563-3a8603aa264c None None] PATH_INFO: `/v3` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:05.176519 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 768/1516] 10.222.0.90 () {54 vars in 925 bytes} [Thu Aug 4 21:23:05 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:23:05.183244 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8836e09c-3134-460b-afe0-42ef6dba01c2 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:05.183671 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8836e09c-3134-460b-afe0-42ef6dba01c2 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:05.183994 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8836e09c-3134-460b-afe0-42ef6dba01c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:05.192006 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f0208d-acfd-4c06-9563-3a8603aa264c None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:05.208438 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f0208d-acfd-4c06-9563-3a8603aa264c None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:05.215921 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3117e6-ff65-40dc-8ba5-3187e149c520 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a2ce1df87fa47e5be301916bf3245fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:05.216705 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3117e6-ff65-40dc-8ba5-3187e149c520 service nova] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:05.217173 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3117e6-ff65-40dc-8ba5-3187e149c520 service nova] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:05.217455 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3117e6-ff65-40dc-8ba5-3187e149c520 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:05.230526 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3117e6-ff65-40dc-8ba5-3187e149c520 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:05.232545 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3117e6-ff65-40dc-8ba5-3187e149c520 service nova] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:05.244677 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 769/1517] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:23:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:23:05.263516 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-8836e09c-3134-460b-afe0-42ef6dba01c2 None None] MFA Rules not processed for user `67d44a0098ab4c36b14b36e9e59ae75c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:23:05.264759 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-8836e09c-3134-460b-afe0-42ef6dba01c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:23:05.352313 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 749/1518] 10.222.0.90 () {58 vars in 1015 bytes} [Thu Aug 4 21:23:05 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:23:09.509483 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b642fac7-d5a3-4fe1-beb1-f039d7b3ccd3 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:09.510062 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b642fac7-d5a3-4fe1-beb1-f039d7b3ccd3 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:09.510174 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b642fac7-d5a3-4fe1-beb1-f039d7b3ccd3 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:09.511622 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 770/1519] 10.222.0.25 () {54 vars in 961 bytes} [Thu Aug 4 21:23:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:23:09.517787 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb5c996-ca96-4652-9cf9-4bfae174a345 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:09.518356 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb5c996-ca96-4652-9cf9-4bfae174a345 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:09.518636 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb5c996-ca96-4652-9cf9-4bfae174a345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:09.539731 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-aeb5c996-ca96-4652-9cf9-4bfae174a345 None None] MFA Rules not processed for user `67d44a0098ab4c36b14b36e9e59ae75c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:23:09.540934 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-aeb5c996-ca96-4652-9cf9-4bfae174a345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:23:09.550540 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 750/1520] 10.222.0.25 () {58 vars in 1058 bytes} [Thu Aug 4 21:23:09 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:23:09.558436 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b642fac7-d5a3-4fe1-beb1-f039d7b3ccd3 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:09.578767 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b642fac7-d5a3-4fe1-beb1-f039d7b3ccd3 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:09.583813 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b9394a-494c-4fa4-a9fd-e04a13bed9c0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67d44a0098ab4c36b14b36e9e59ae75c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:09.584436 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b9394a-494c-4fa4-a9fd-e04a13bed9c0 service placement] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:09.584721 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b9394a-494c-4fa4-a9fd-e04a13bed9c0 service placement] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:09.585005 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b9394a-494c-4fa4-a9fd-e04a13bed9c0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:09.594867 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b9394a-494c-4fa4-a9fd-e04a13bed9c0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:09.596551 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b9394a-494c-4fa4-a9fd-e04a13bed9c0 service placement] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:09.607294 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 771/1521] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:23:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:23:16.537099 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977ea6bb-f834-49e3-a7ab-b7cb9400e694 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:16.537707 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977ea6bb-f834-49e3-a7ab-b7cb9400e694 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:16.537707 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977ea6bb-f834-49e3-a7ab-b7cb9400e694 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:16.539029 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 751/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 21:23:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:23:16.544075 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57987ce5-a139-4586-b8e9-e0478eff014a None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:16.544597 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57987ce5-a139-4586-b8e9-e0478eff014a None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:16.544705 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57987ce5-a139-4586-b8e9-e0478eff014a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:16.565116 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-57987ce5-a139-4586-b8e9-e0478eff014a None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:23:16.566372 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-57987ce5-a139-4586-b8e9-e0478eff014a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:23:16.574453 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 772/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 21:23:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:23:16.732599 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977ea6bb-f834-49e3-a7ab-b7cb9400e694 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:16.748325 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977ea6bb-f834-49e3-a7ab-b7cb9400e694 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:16.754660 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c8ca0f-2408-4f13-bfbf-e645374097b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a2ce1df87fa47e5be301916bf3245fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:16.755275 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c8ca0f-2408-4f13-bfbf-e645374097b0 service nova] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:16.755585 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c8ca0f-2408-4f13-bfbf-e645374097b0 service nova] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:16.755855 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c8ca0f-2408-4f13-bfbf-e645374097b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:16.762484 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:16.765133 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c8ca0f-2408-4f13-bfbf-e645374097b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:16.770262 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c8ca0f-2408-4f13-bfbf-e645374097b0 service nova] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:16.777657 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 752/1524] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:23:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:23:17.623363 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98df1ecf-8943-4b13-9a28-60ee3a1bea00 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:17.624002 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98df1ecf-8943-4b13-9a28-60ee3a1bea00 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:17.624002 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98df1ecf-8943-4b13-9a28-60ee3a1bea00 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:17.625273 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 773/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 21:23:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:23:17.630567 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc2fdca-3da4-434c-941a-e5e3e079eecf None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:17.630993 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc2fdca-3da4-434c-941a-e5e3e079eecf None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:17.631114 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc2fdca-3da4-434c-941a-e5e3e079eecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:17.651634 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-3fc2fdca-3da4-434c-941a-e5e3e079eecf None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:23:17.652820 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-3fc2fdca-3da4-434c-941a-e5e3e079eecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:23:17.667930 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 753/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 21:23:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:23:17.823576 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98df1ecf-8943-4b13-9a28-60ee3a1bea00 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:17.839452 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98df1ecf-8943-4b13-9a28-60ee3a1bea00 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:17.845505 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72513f5-4f87-413a-815c-270e83943cc2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a2ce1df87fa47e5be301916bf3245fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:17.846132 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72513f5-4f87-413a-815c-270e83943cc2 service nova] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:17.846415 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72513f5-4f87-413a-815c-270e83943cc2 service nova] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:17.846688 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72513f5-4f87-413a-815c-270e83943cc2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:17.856704 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72513f5-4f87-413a-815c-270e83943cc2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:17.858613 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72513f5-4f87-413a-815c-270e83943cc2 service nova] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:17.869402 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 774/1527] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:23:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:23:22.003017 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6612a39c-5dd9-4a2c-806a-7f79faac63a9 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:22.003975 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6612a39c-5dd9-4a2c-806a-7f79faac63a9 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:22.003975 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6612a39c-5dd9-4a2c-806a-7f79faac63a9 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:22.005035 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 754/1528] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 21:23:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:23:22.010728 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1c6a4e-ea57-4c9b-a15e-99a58d114b2e None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:22.011239 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1c6a4e-ea57-4c9b-a15e-99a58d114b2e None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:22.011387 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1c6a4e-ea57-4c9b-a15e-99a58d114b2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:22.037001 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-1f1c6a4e-ea57-4c9b-a15e-99a58d114b2e None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:23:22.038313 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-1f1c6a4e-ea57-4c9b-a15e-99a58d114b2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:23:22.046732 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 775/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 21:23:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:23:22.128311 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6612a39c-5dd9-4a2c-806a-7f79faac63a9 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:22.130046 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-6612a39c-5dd9-4a2c-806a-7f79faac63a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:23:22.200891 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6612a39c-5dd9-4a2c-806a-7f79faac63a9 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:22.205100 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295276f7-5fcf-4261-8052-62851c1fba68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:22.205731 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295276f7-5fcf-4261-8052-62851c1fba68 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:22.206017 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295276f7-5fcf-4261-8052-62851c1fba68 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:22.206294 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295276f7-5fcf-4261-8052-62851c1fba68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:22.216766 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295276f7-5fcf-4261-8052-62851c1fba68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:22.225053 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295276f7-5fcf-4261-8052-62851c1fba68 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:22.237888 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 755/1530] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:23:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:23:24.335571 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170568a1-5946-4745-b86f-f355b3ba7e0e None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:24.336421 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170568a1-5946-4745-b86f-f355b3ba7e0e None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:24.336593 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170568a1-5946-4745-b86f-f355b3ba7e0e None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:24.344820 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 776/1531] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 21:23:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:23:24.345084 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab2be2e-63c3-4a66-a75d-0d7781ec5d35 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:24.345808 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab2be2e-63c3-4a66-a75d-0d7781ec5d35 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:24.345982 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab2be2e-63c3-4a66-a75d-0d7781ec5d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:24.368263 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-9ab2be2e-63c3-4a66-a75d-0d7781ec5d35 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:23:24.369497 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-9ab2be2e-63c3-4a66-a75d-0d7781ec5d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:23:24.380655 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 756/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 21:23:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:23:24.416774 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170568a1-5946-4745-b86f-f355b3ba7e0e None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:24.429467 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170568a1-5946-4745-b86f-f355b3ba7e0e None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:24.434692 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c733cf19-4617-41d3-ae73-65acf6e91e3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:24.435662 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c733cf19-4617-41d3-ae73-65acf6e91e3c service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:24.435953 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c733cf19-4617-41d3-ae73-65acf6e91e3c service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:24.436233 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c733cf19-4617-41d3-ae73-65acf6e91e3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:24.447260 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c733cf19-4617-41d3-ae73-65acf6e91e3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:24.449710 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c733cf19-4617-41d3-ae73-65acf6e91e3c service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:24.459423 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 777/1533] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:23:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:23:28.233847 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faae25f-ae7b-41ae-a249-904ecb0f06b1 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:28.234351 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faae25f-ae7b-41ae-a249-904ecb0f06b1 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:28.234446 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faae25f-ae7b-41ae-a249-904ecb0f06b1 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:28.235857 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 757/1534] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 21:23:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:23:28.241557 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf87fe9-05c2-4957-b550-e20744574dad None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:28.242048 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf87fe9-05c2-4957-b550-e20744574dad None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:28.242170 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf87fe9-05c2-4957-b550-e20744574dad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:28.255506 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=58 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:28.271628 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-aaf87fe9-05c2-4957-b550-e20744574dad None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:23:28.272864 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-aaf87fe9-05c2-4957-b550-e20744574dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:23:28.281557 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 778/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 21:23:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:23:28.600243 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faae25f-ae7b-41ae-a249-904ecb0f06b1 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:28.601914 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-2faae25f-ae7b-41ae-a249-904ecb0f06b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:23:28.690227 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faae25f-ae7b-41ae-a249-904ecb0f06b1 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:28.693860 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fa02e6-b04c-4233-8fc5-4b1e3f7a6453 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e31f2728056b460fa474478eb85ca1d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:28.694505 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fa02e6-b04c-4233-8fc5-4b1e3f7a6453 service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:28.694784 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fa02e6-b04c-4233-8fc5-4b1e3f7a6453 service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:28.695059 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fa02e6-b04c-4233-8fc5-4b1e3f7a6453 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:28.704998 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fa02e6-b04c-4233-8fc5-4b1e3f7a6453 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:28.706460 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fa02e6-b04c-4233-8fc5-4b1e3f7a6453 service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:28.717124 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 758/1536] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:23:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:23:30.854869 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbdb35a-1533-4de3-8395-88c975a2f542 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:30.855490 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbdb35a-1533-4de3-8395-88c975a2f542 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:30.855490 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbdb35a-1533-4de3-8395-88c975a2f542 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:30.856811 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 779/1537] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 21:23:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:23:30.863502 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a998adcf-2eda-4542-8f4f-30ad50bee9ff None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:30.864093 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a998adcf-2eda-4542-8f4f-30ad50bee9ff None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:30.864093 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a998adcf-2eda-4542-8f4f-30ad50bee9ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:30.884247 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-a998adcf-2eda-4542-8f4f-30ad50bee9ff None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:23:30.885455 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-a998adcf-2eda-4542-8f4f-30ad50bee9ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:23:30.893785 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 759/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 21:23:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:23:31.220241 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbdb35a-1533-4de3-8395-88c975a2f542 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:31.221161 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-dbbdb35a-1533-4de3-8395-88c975a2f542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:23:31.238208 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbdb35a-1533-4de3-8395-88c975a2f542 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:31.244025 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136c38f4-a87e-49c3-8733-896b500ec075 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e31f2728056b460fa474478eb85ca1d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:31.244843 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136c38f4-a87e-49c3-8733-896b500ec075 service cinder] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:31.245121 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136c38f4-a87e-49c3-8733-896b500ec075 service cinder] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:31.245398 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136c38f4-a87e-49c3-8733-896b500ec075 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:31.255567 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136c38f4-a87e-49c3-8733-896b500ec075 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:31.257086 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136c38f4-a87e-49c3-8733-896b500ec075 service cinder] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:31.268534 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 780/1539] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:23:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:23:40.879284 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=63 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:41.262952 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=8 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:24:57.792030 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ecd583-724d-4b63-be10-58a002e270f3 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:24:57.793402 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ecd583-724d-4b63-be10-58a002e270f3 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:24:57.793402 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ecd583-724d-4b63-be10-58a002e270f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:24:57.817964 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-75ecd583-724d-4b63-be10-58a002e270f3 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:24:57.819175 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-75ecd583-724d-4b63-be10-58a002e270f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:24:57.831480 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 760/1540] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:24:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:24:57.841588 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0c2d79-839f-4d79-b19d-44218efacd84 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:24:57.842825 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0c2d79-839f-4d79-b19d-44218efacd84 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:24:57.843556 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0c2d79-839f-4d79-b19d-44218efacd84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:24:57.880021 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-df0c2d79-839f-4d79-b19d-44218efacd84 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:24:57.881649 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-df0c2d79-839f-4d79-b19d-44218efacd84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:24:57.891131 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 781/1541] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:24:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:24:57.896550 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ecd583-724d-4b63-be10-58a002e270f3 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:24:57.911548 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ecd583-724d-4b63-be10-58a002e270f3 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:24:57.919486 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e101caf0-34c7-47e0-b48e-5ca5662b025e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:24:57.920140 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e101caf0-34c7-47e0-b48e-5ca5662b025e admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:24:57.920424 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e101caf0-34c7-47e0-b48e-5ca5662b025e admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:24:57.920702 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e101caf0-34c7-47e0-b48e-5ca5662b025e admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:24:57.921410 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e101caf0-34c7-47e0-b48e-5ca5662b025e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:24:57.921712 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e101caf0-34c7-47e0-b48e-5ca5662b025e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:24:57.922747 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: WARNING py.warnings [None req-e101caf0-34c7-47e0-b48e-5ca5662b025e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 04 21:24:57.922747 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: warnings.warn(msg) Aug 04 21:24:57.922747 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: Aug 04 21:24:57.924219 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e101caf0-34c7-47e0-b48e-5ca5662b025e admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:24:57.943034 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 761/1542] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Aug 4 21:24:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:24:57.951749 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0c2d79-839f-4d79-b19d-44218efacd84 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:24:57.969959 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0c2d79-839f-4d79-b19d-44218efacd84 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:24:57.979584 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44f3c1c-ae9e-42c7-9dac-4b04aa1e5f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:24:57.980694 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f3c1c-ae9e-42c7-9dac-4b04aa1e5f9c admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:24:57.981209 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f3c1c-ae9e-42c7-9dac-4b04aa1e5f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:24:57.981973 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f3c1c-ae9e-42c7-9dac-4b04aa1e5f9c admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:24:57.982906 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44f3c1c-ae9e-42c7-9dac-4b04aa1e5f9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:24:57.983415 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07173af-2127-46da-9148-e7e0d08ece2c None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:24:57.983709 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07173af-2127-46da-9148-e7e0d08ece2c None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:24:57.983983 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07173af-2127-46da-9148-e7e0d08ece2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:24:57.984932 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: WARNING py.warnings [None req-f44f3c1c-ae9e-42c7-9dac-4b04aa1e5f9c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 21:24:57.984932 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: warnings.warn(msg) Aug 04 21:24:57.984932 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: Aug 04 21:24:57.987075 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44f3c1c-ae9e-42c7-9dac-4b04aa1e5f9c admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:24:58.028491 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-f07173af-2127-46da-9148-e7e0d08ece2c None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:24:58.030660 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-f07173af-2127-46da-9148-e7e0d08ece2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:24:58.049545 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 762/1543] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:24:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:24:58.060066 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0803859-c9eb-4094-826d-03c8baaec35d None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:24:58.060360 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0803859-c9eb-4094-826d-03c8baaec35d None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:24:58.060621 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0803859-c9eb-4094-826d-03c8baaec35d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:24:58.101028 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-a0803859-c9eb-4094-826d-03c8baaec35d None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:24:58.102453 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-a0803859-c9eb-4094-826d-03c8baaec35d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:24:58.115553 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 763/1544] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:24:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:24:58.120502 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0803859-c9eb-4094-826d-03c8baaec35d None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:24:58.133362 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0803859-c9eb-4094-826d-03c8baaec35d None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:24:58.145402 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba43f20-640c-4ad1-a80b-f5debcbd2f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:24:58.146285 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba43f20-640c-4ad1-a80b-f5debcbd2f21 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:24:58.146579 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba43f20-640c-4ad1-a80b-f5debcbd2f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:24:58.146862 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba43f20-640c-4ad1-a80b-f5debcbd2f21 admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:24:58.147608 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba43f20-640c-4ad1-a80b-f5debcbd2f21 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:24:58.148394 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba43f20-640c-4ad1-a80b-f5debcbd2f21 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:24:58.149406 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 782/1545] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:24:57 2022] POST /identity/v3/projects => generated 385 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:24:58.149594 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba43f20-640c-4ad1-a80b-f5debcbd2f21 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:24:58.168948 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44f3c1c-ae9e-42c7-9dac-4b04aa1e5f9c admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:24:58.178794 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 764/1546] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Aug 4 21:24:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:24:58.188180 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba43f20-640c-4ad1-a80b-f5debcbd2f21 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:24:58.206183 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba43f20-640c-4ad1-a80b-f5debcbd2f21 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:24:58.337839 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44f3c1c-ae9e-42c7-9dac-4b04aa1e5f9c admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:24:58.352205 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2237e3c4-484a-4f5a-a65f-f4769dc2bd10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:24:58.352960 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2237e3c4-484a-4f5a-a65f-f4769dc2bd10 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:24:58.353249 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2237e3c4-484a-4f5a-a65f-f4769dc2bd10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:24:58.353719 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2237e3c4-484a-4f5a-a65f-f4769dc2bd10 admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:24:58.355037 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2237e3c4-484a-4f5a-a65f-f4769dc2bd10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:24:58.356379 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e01191-70fd-46db-aa02-a2b3a5bfb949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:24:58.357395 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2237e3c4-484a-4f5a-a65f-f4769dc2bd10 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:24:58.357672 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e01191-70fd-46db-aa02-a2b3a5bfb949 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:24:58.358149 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e01191-70fd-46db-aa02-a2b3a5bfb949 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:24:58.358592 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e01191-70fd-46db-aa02-a2b3a5bfb949 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:24:58.359676 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e01191-70fd-46db-aa02-a2b3a5bfb949 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:24:58.363506 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e01191-70fd-46db-aa02-a2b3a5bfb949 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:24:58.475994 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 783/1547] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:24:58 2022] POST /identity/v3/users => generated 293 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:24:58.477645 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786c7ccd-76a2-46d0-a014-e88f07c5ea89 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:24:58.477953 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786c7ccd-76a2-46d0-a014-e88f07c5ea89 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:24:58.478233 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786c7ccd-76a2-46d0-a014-e88f07c5ea89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:24:58.506928 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 765/1548] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:24:58 2022] POST /identity/v3/projects => generated 393 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:24:58.508411 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3f67f2-f3d2-4534-a399-464648c6d46f None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:24:58.508631 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-786c7ccd-76a2-46d0-a014-e88f07c5ea89 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:24:58.508748 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3f67f2-f3d2-4534-a399-464648c6d46f None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:24:58.508963 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3f67f2-f3d2-4534-a399-464648c6d46f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:24:58.509841 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-786c7ccd-76a2-46d0-a014-e88f07c5ea89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:24:58.550652 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-bc3f67f2-f3d2-4534-a399-464648c6d46f None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:24:58.551919 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-bc3f67f2-f3d2-4534-a399-464648c6d46f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:24:58.649821 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 784/1549] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:24:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:24:58.650465 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786c7ccd-76a2-46d0-a014-e88f07c5ea89 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:24:58.665483 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 766/1550] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:24:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:24:58.666674 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3f67f2-f3d2-4534-a399-464648c6d46f None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:24:58.667343 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786c7ccd-76a2-46d0-a014-e88f07c5ea89 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:24:58.672401 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57beea6e-374b-44d3-b080-c47c1bab460d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:24:58.673038 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57beea6e-374b-44d3-b080-c47c1bab460d admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:24:58.673385 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57beea6e-374b-44d3-b080-c47c1bab460d admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:24:58.673787 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57beea6e-374b-44d3-b080-c47c1bab460d admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:24:58.674856 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57beea6e-374b-44d3-b080-c47c1bab460d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:24:58.677210 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57beea6e-374b-44d3-b080-c47c1bab460d admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:24:58.679026 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3f67f2-f3d2-4534-a399-464648c6d46f None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:24:58.684239 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76be4e0c-9bc9-4261-83a8-7fe06efb5e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:24:58.684864 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76be4e0c-9bc9-4261-83a8-7fe06efb5e29 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:24:58.685151 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76be4e0c-9bc9-4261-83a8-7fe06efb5e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:24:58.685562 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76be4e0c-9bc9-4261-83a8-7fe06efb5e29 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:24:58.686074 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76be4e0c-9bc9-4261-83a8-7fe06efb5e29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:24:58.687597 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: WARNING py.warnings [None req-76be4e0c-9bc9-4261-83a8-7fe06efb5e29 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 21:24:58.687597 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: warnings.warn(msg) Aug 04 21:24:58.687597 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: Aug 04 21:24:58.688996 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76be4e0c-9bc9-4261-83a8-7fe06efb5e29 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:24:58.692625 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 785/1551] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:24:58 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:24:58.692625 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa13ef76-d4e2-4a0e-b32b-d0d7664fe794 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:24:58.692625 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa13ef76-d4e2-4a0e-b32b-d0d7664fe794 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:24:58.692625 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa13ef76-d4e2-4a0e-b32b-d0d7664fe794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:24:58.716381 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-aa13ef76-d4e2-4a0e-b32b-d0d7664fe794 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:24:58.717823 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-aa13ef76-d4e2-4a0e-b32b-d0d7664fe794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:24:58.728130 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 786/1552] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:24:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:24:58.729633 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e327d-b881-49d8-a3a3-cfc149cdb01e None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:24:58.729921 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e327d-b881-49d8-a3a3-cfc149cdb01e None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:24:58.730252 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e327d-b881-49d8-a3a3-cfc149cdb01e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:24:58.749808 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-d75e327d-b881-49d8-a3a3-cfc149cdb01e None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:24:58.751015 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-d75e327d-b881-49d8-a3a3-cfc149cdb01e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:24:58.763373 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 787/1553] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:24:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:24:58.764680 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1108bc-42b9-418b-aada-4cf06d110bad None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:24:58.764963 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1108bc-42b9-418b-aada-4cf06d110bad None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:24:58.765233 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1108bc-42b9-418b-aada-4cf06d110bad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:24:58.788181 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-cd1108bc-42b9-418b-aada-4cf06d110bad None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:24:58.789466 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-cd1108bc-42b9-418b-aada-4cf06d110bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:24:58.797774 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 788/1554] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:24:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:24:58.801084 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae72c147-d9e1-420e-a40c-749aff7fe19a None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:24:58.801084 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae72c147-d9e1-420e-a40c-749aff7fe19a None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:24:58.801084 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae72c147-d9e1-420e-a40c-749aff7fe19a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:24:58.821484 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-ae72c147-d9e1-420e-a40c-749aff7fe19a None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:24:58.824091 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-ae72c147-d9e1-420e-a40c-749aff7fe19a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:24:58.835226 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 789/1555] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:24:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:24:58.835226 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69285b78-65e1-40f1-8a0b-f3c8f830d665 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:24:58.835226 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69285b78-65e1-40f1-8a0b-f3c8f830d665 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:24:58.835226 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69285b78-65e1-40f1-8a0b-f3c8f830d665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:24:58.847366 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 767/1556] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:24:58 2022] POST /identity/v3/users => generated 297 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:24:58.848429 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76be4e0c-9bc9-4261-83a8-7fe06efb5e29 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:24:58.859852 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-69285b78-65e1-40f1-8a0b-f3c8f830d665 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:24:58.860439 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76be4e0c-9bc9-4261-83a8-7fe06efb5e29 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:24:58.861958 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-69285b78-65e1-40f1-8a0b-f3c8f830d665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:24:58.865286 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7559323-b16d-4d69-a3f1-982869ad35aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:24:58.865914 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7559323-b16d-4d69-a3f1-982869ad35aa admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:24:58.866194 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7559323-b16d-4d69-a3f1-982869ad35aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:24:58.866464 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7559323-b16d-4d69-a3f1-982869ad35aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:24:58.867188 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7559323-b16d-4d69-a3f1-982869ad35aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:24:58.868880 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7559323-b16d-4d69-a3f1-982869ad35aa admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:24:58.879092 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 790/1557] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:24:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:24:58.881173 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b79b92d-7a86-43c1-8c3d-7d1e8d97d380 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:24:58.885172 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 768/1558] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:24:58 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:24:58.885172 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46de2b9-acf3-449e-ac2c-6a23aea4322c None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:24:58.885172 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46de2b9-acf3-449e-ac2c-6a23aea4322c None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:24:58.885172 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46de2b9-acf3-449e-ac2c-6a23aea4322c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:24:58.885990 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b79b92d-7a86-43c1-8c3d-7d1e8d97d380 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:24:58.886622 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b79b92d-7a86-43c1-8c3d-7d1e8d97d380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:24:58.911277 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-4b79b92d-7a86-43c1-8c3d-7d1e8d97d380 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:24:58.912911 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-4b79b92d-7a86-43c1-8c3d-7d1e8d97d380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:24:58.913928 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-e46de2b9-acf3-449e-ac2c-6a23aea4322c None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:24:58.916782 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-e46de2b9-acf3-449e-ac2c-6a23aea4322c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:24:58.925179 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 769/1559] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:24:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:24:58.925856 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b27d7c7-e41a-43d0-b790-26119ddf67f6 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:24:58.926147 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b27d7c7-e41a-43d0-b790-26119ddf67f6 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:24:58.926420 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b27d7c7-e41a-43d0-b790-26119ddf67f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:24:58.930848 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 791/1560] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:24:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:24:58.933220 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ce665-ad74-4f5d-8b99-f53f15255ac6 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:24:58.933515 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ce665-ad74-4f5d-8b99-f53f15255ac6 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:24:58.933794 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ce665-ad74-4f5d-8b99-f53f15255ac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:24:58.952991 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:24:58.961834 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-6b27d7c7-e41a-43d0-b790-26119ddf67f6 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:24:58.963342 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-6b27d7c7-e41a-43d0-b790-26119ddf67f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:24:58.965597 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-226ce665-ad74-4f5d-8b99-f53f15255ac6 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:24:58.966819 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-226ce665-ad74-4f5d-8b99-f53f15255ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:24:58.976133 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 770/1561] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:24:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:24:58.977374 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b27d7c7-e41a-43d0-b790-26119ddf67f6 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:24:58.982776 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 792/1562] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:24:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:24:58.983690 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226ce665-ad74-4f5d-8b99-f53f15255ac6 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:24:58.990618 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:24:58.994038 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b27d7c7-e41a-43d0-b790-26119ddf67f6 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:24:58.994468 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226ce665-ad74-4f5d-8b99-f53f15255ac6 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:24:58.999876 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c727e58d-f548-46ee-b8f0-92cb910269e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:24:59.000547 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c727e58d-f548-46ee-b8f0-92cb910269e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:24:59.000830 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c727e58d-f548-46ee-b8f0-92cb910269e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:24:59.001102 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c727e58d-f548-46ee-b8f0-92cb910269e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:24:59.002346 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c727e58d-f548-46ee-b8f0-92cb910269e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:24:59.002588 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2241d077-7030-41d9-a74b-cdd6763c752a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:24:59.003090 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2241d077-7030-41d9-a74b-cdd6763c752a admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:24:59.003436 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2241d077-7030-41d9-a74b-cdd6763c752a admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:24:59.007800 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2241d077-7030-41d9-a74b-cdd6763c752a admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:24:59.007925 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c727e58d-f548-46ee-b8f0-92cb910269e8 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:24:59.008023 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2241d077-7030-41d9-a74b-cdd6763c752a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:24:59.008023 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2241d077-7030-41d9-a74b-cdd6763c752a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:24:59.008023 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2241d077-7030-41d9-a74b-cdd6763c752a admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:24:59.019471 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 793/1563] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:24:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:24:59.019471 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c727e58d-f548-46ee-b8f0-92cb910269e8 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:24:59.019746 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 771/1564] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Aug 4 21:24:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:24:59.020181 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2241d077-7030-41d9-a74b-cdd6763c752a admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:24:59.028934 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c727e58d-f548-46ee-b8f0-92cb910269e8 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:24:59.034267 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bb1fcd-5d68-433a-aad0-327ec4ec75d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:24:59.034899 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bb1fcd-5d68-433a-aad0-327ec4ec75d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:24:59.035343 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bb1fcd-5d68-433a-aad0-327ec4ec75d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:24:59.035650 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bb1fcd-5d68-433a-aad0-327ec4ec75d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:24:59.036041 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2241d077-7030-41d9-a74b-cdd6763c752a admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:24:59.036361 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bb1fcd-5d68-433a-aad0-327ec4ec75d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:24:59.036827 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bb1fcd-5d68-433a-aad0-327ec4ec75d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:24:59.038963 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bb1fcd-5d68-433a-aad0-327ec4ec75d2 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:24:59.040783 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45425ab-f04e-45ec-9980-d8790e791473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:24:59.041660 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45425ab-f04e-45ec-9980-d8790e791473 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:24:59.041947 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45425ab-f04e-45ec-9980-d8790e791473 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:24:59.042904 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45425ab-f04e-45ec-9980-d8790e791473 admin admin] PATH_INFO: `/v3/projects/f28e5935db5c4be4871d727e1958424b/users/ea2f8480bc7d44f6a51e4a1d875a1c7c/roles/8feebc60363c4feb870718b1721e5324` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:24:59.052965 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 794/1565] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Aug 4 21:24:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:24:59.053902 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bb1fcd-5d68-433a-aad0-327ec4ec75d2 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:24:59.058520 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45425ab-f04e-45ec-9980-d8790e791473 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f28e5935db5c4be4871d727e1958424b, user_id=ea2f8480bc7d44f6a51e4a1d875a1c7c, role_id=8feebc60363c4feb870718b1721e5324)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:24:59.061296 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45425ab-f04e-45ec-9980-d8790e791473 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:24:59.065936 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bb1fcd-5d68-433a-aad0-327ec4ec75d2 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:24:59.070882 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88681d2-50ff-402c-905d-e92d1d282b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:24:59.071553 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88681d2-50ff-402c-905d-e92d1d282b47 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:24:59.071841 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88681d2-50ff-402c-905d-e92d1d282b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:24:59.072122 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88681d2-50ff-402c-905d-e92d1d282b47 admin admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:24:59.072829 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88681d2-50ff-402c-905d-e92d1d282b47 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:24:59.073139 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88681d2-50ff-402c-905d-e92d1d282b47 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:24:59.075076 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88681d2-50ff-402c-905d-e92d1d282b47 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:24:59.090114 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 795/1566] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Aug 4 21:24:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:24:59.091562 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e8f2b2-5d1e-4a5b-ab73-6c7c7a15d277 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:24:59.091859 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e8f2b2-5d1e-4a5b-ab73-6c7c7a15d277 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:24:59.092139 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e8f2b2-5d1e-4a5b-ab73-6c7c7a15d277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:24:59.112847 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-b6e8f2b2-5d1e-4a5b-ab73-6c7c7a15d277 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:24:59.114073 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-b6e8f2b2-5d1e-4a5b-ab73-6c7c7a15d277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:24:59.122182 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 796/1567] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:24:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:24:59.123182 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e8f2b2-5d1e-4a5b-ab73-6c7c7a15d277 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:24:59.133642 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e8f2b2-5d1e-4a5b-ab73-6c7c7a15d277 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:24:59.138632 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47724143-b584-43c6-aa51-213beb2f70c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:24:59.139258 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47724143-b584-43c6-aa51-213beb2f70c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:24:59.139584 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47724143-b584-43c6-aa51-213beb2f70c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:24:59.139865 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47724143-b584-43c6-aa51-213beb2f70c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:24:59.140569 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47724143-b584-43c6-aa51-213beb2f70c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:24:59.140876 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47724143-b584-43c6-aa51-213beb2f70c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:24:59.142531 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47724143-b584-43c6-aa51-213beb2f70c9 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:24:59.152146 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 772/1568] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:24:59 2022] PUT /identity/v3/projects/f28e5935db5c4be4871d727e1958424b/users/ea2f8480bc7d44f6a51e4a1d875a1c7c/roles/8feebc60363c4feb870718b1721e5324 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:24:59.153452 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45425ab-f04e-45ec-9980-d8790e791473 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:24:59.154912 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 797/1569] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Aug 4 21:24:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:24:59.156032 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47724143-b584-43c6-aa51-213beb2f70c9 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:24:59.176931 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47724143-b584-43c6-aa51-213beb2f70c9 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:24:59.233961 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45425ab-f04e-45ec-9980-d8790e791473 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:24:59.239786 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34afcea7-dcd1-40f6-a5d1-bc7dd0f28174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:24:59.240416 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34afcea7-dcd1-40f6-a5d1-bc7dd0f28174 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:24:59.240695 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34afcea7-dcd1-40f6-a5d1-bc7dd0f28174 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:24:59.240964 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34afcea7-dcd1-40f6-a5d1-bc7dd0f28174 admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:24:59.241764 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34afcea7-dcd1-40f6-a5d1-bc7dd0f28174 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:24:59.242066 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34afcea7-dcd1-40f6-a5d1-bc7dd0f28174 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:24:59.243920 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34afcea7-dcd1-40f6-a5d1-bc7dd0f28174 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:24:59.247073 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5014b194-1067-403e-ac1d-cfc02ad96c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:24:59.247910 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5014b194-1067-403e-ac1d-cfc02ad96c65 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:24:59.248313 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5014b194-1067-403e-ac1d-cfc02ad96c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:24:59.248683 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5014b194-1067-403e-ac1d-cfc02ad96c65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:24:59.249529 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5014b194-1067-403e-ac1d-cfc02ad96c65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:24:59.251437 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5014b194-1067-403e-ac1d-cfc02ad96c65 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:24:59.258206 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 773/1570] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Aug 4 21:24:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:24:59.259037 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34afcea7-dcd1-40f6-a5d1-bc7dd0f28174 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:24:59.270716 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34afcea7-dcd1-40f6-a5d1-bc7dd0f28174 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:24:59.270933 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 798/1571] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:24:59 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:24:59.271857 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5014b194-1067-403e-ac1d-cfc02ad96c65 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:24:59.275627 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881922c4-bf17-47f0-a772-bd00ca9fa0d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:24:59.276253 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881922c4-bf17-47f0-a772-bd00ca9fa0d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:24:59.276624 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881922c4-bf17-47f0-a772-bd00ca9fa0d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:24:59.276915 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881922c4-bf17-47f0-a772-bd00ca9fa0d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:24:59.277583 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881922c4-bf17-47f0-a772-bd00ca9fa0d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:24:59.279609 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881922c4-bf17-47f0-a772-bd00ca9fa0d2 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:24:59.283342 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5014b194-1067-403e-ac1d-cfc02ad96c65 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:24:59.287925 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ebc4ad-0d3c-4045-bf76-f151d37f5951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:24:59.288547 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ebc4ad-0d3c-4045-bf76-f151d37f5951 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:24:59.288837 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ebc4ad-0d3c-4045-bf76-f151d37f5951 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:24:59.289099 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ebc4ad-0d3c-4045-bf76-f151d37f5951 admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:24:59.289770 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ebc4ad-0d3c-4045-bf76-f151d37f5951 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:24:59.291712 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ebc4ad-0d3c-4045-bf76-f151d37f5951 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:24:59.380015 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 774/1572] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:24:59 2022] POST /identity/v3/projects => generated 377 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:24:59.381262 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881922c4-bf17-47f0-a772-bd00ca9fa0d2 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:24:59.409581 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 799/1573] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:24:59 2022] POST /identity/v3/projects => generated 379 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:24:59.409581 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ebc4ad-0d3c-4045-bf76-f151d37f5951 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:24:59.459672 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881922c4-bf17-47f0-a772-bd00ca9fa0d2 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:24:59.476737 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ebc4ad-0d3c-4045-bf76-f151d37f5951 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:24:59.481781 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c804f5-661f-4f7d-aa24-9627ac09ebed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:24:59.482436 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c804f5-661f-4f7d-aa24-9627ac09ebed admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:24:59.482728 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c804f5-661f-4f7d-aa24-9627ac09ebed admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:24:59.483002 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c804f5-661f-4f7d-aa24-9627ac09ebed admin admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:24:59.483755 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c804f5-661f-4f7d-aa24-9627ac09ebed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:24:59.484066 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c804f5-661f-4f7d-aa24-9627ac09ebed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:24:59.485708 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c804f5-661f-4f7d-aa24-9627ac09ebed admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:24:59.499480 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 800/1574] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Aug 4 21:24:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:24:59.501146 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56544b1-f592-419b-bbbc-cc8af1c271c5 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:24:59.501599 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56544b1-f592-419b-bbbc-cc8af1c271c5 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:24:59.501979 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56544b1-f592-419b-bbbc-cc8af1c271c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:24:59.509435 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:24:59.522297 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-b56544b1-f592-419b-bbbc-cc8af1c271c5 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:24:59.523581 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-b56544b1-f592-419b-bbbc-cc8af1c271c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:24:59.536012 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 801/1575] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:24:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:24:59.536926 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56544b1-f592-419b-bbbc-cc8af1c271c5 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:24:59.537327 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85df6f1-f11c-4fc9-8cfc-6d0ecfc504ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:24:59.538021 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85df6f1-f11c-4fc9-8cfc-6d0ecfc504ad admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:24:59.538303 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85df6f1-f11c-4fc9-8cfc-6d0ecfc504ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:24:59.538574 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85df6f1-f11c-4fc9-8cfc-6d0ecfc504ad admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:24:59.539251 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85df6f1-f11c-4fc9-8cfc-6d0ecfc504ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:24:59.541450 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85df6f1-f11c-4fc9-8cfc-6d0ecfc504ad admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:24:59.551638 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56544b1-f592-419b-bbbc-cc8af1c271c5 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:24:59.562737 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194ff407-89a5-4f77-8d12-6bf880358ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:24:59.563487 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194ff407-89a5-4f77-8d12-6bf880358ac0 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:24:59.563762 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194ff407-89a5-4f77-8d12-6bf880358ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:24:59.564084 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194ff407-89a5-4f77-8d12-6bf880358ac0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:24:59.564763 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194ff407-89a5-4f77-8d12-6bf880358ac0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:24:59.566759 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194ff407-89a5-4f77-8d12-6bf880358ac0 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:24:59.572563 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:24:59.681928 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 775/1576] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:24:59 2022] POST /identity/v3/projects => generated 403 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:24:59.682742 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85df6f1-f11c-4fc9-8cfc-6d0ecfc504ad admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:24:59.705519 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 802/1577] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:24:59 2022] POST /identity/v3/projects => generated 383 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:24:59.705519 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194ff407-89a5-4f77-8d12-6bf880358ac0 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:24:59.760186 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85df6f1-f11c-4fc9-8cfc-6d0ecfc504ad admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:24:59.775954 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194ff407-89a5-4f77-8d12-6bf880358ac0 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:24:59.780928 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265caca3-325a-484e-9f02-2da31e17eb58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:24:59.781772 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265caca3-325a-484e-9f02-2da31e17eb58 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:24:59.782047 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265caca3-325a-484e-9f02-2da31e17eb58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:24:59.782318 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265caca3-325a-484e-9f02-2da31e17eb58 admin admin] PATH_INFO: `/v3/projects/0cc9fe2e0830451c823180afe60ec921/users/3d5f53509b2f42269cd26e77c79ad8be/roles/8feebc60363c4feb870718b1721e5324` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:24:59.800212 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265caca3-325a-484e-9f02-2da31e17eb58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cc9fe2e0830451c823180afe60ec921, user_id=3d5f53509b2f42269cd26e77c79ad8be, role_id=8feebc60363c4feb870718b1721e5324)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:24:59.801268 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: WARNING py.warnings [None req-265caca3-325a-484e-9f02-2da31e17eb58 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 21:24:59.801268 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: warnings.warn(msg) Aug 04 21:24:59.801268 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: Aug 04 21:24:59.803384 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265caca3-325a-484e-9f02-2da31e17eb58 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:24:59.828470 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab50ac43-cb38-4d80-ae45-0cf58b46c483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:24:59.829194 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab50ac43-cb38-4d80-ae45-0cf58b46c483 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:24:59.829473 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab50ac43-cb38-4d80-ae45-0cf58b46c483 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:24:59.829776 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab50ac43-cb38-4d80-ae45-0cf58b46c483 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:24:59.830443 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab50ac43-cb38-4d80-ae45-0cf58b46c483 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:24:59.832423 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab50ac43-cb38-4d80-ae45-0cf58b46c483 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:24:59.875764 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 803/1578] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:24:59 2022] PUT /identity/v3/projects/0cc9fe2e0830451c823180afe60ec921/users/3d5f53509b2f42269cd26e77c79ad8be/roles/8feebc60363c4feb870718b1721e5324 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:24:59.876651 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265caca3-325a-484e-9f02-2da31e17eb58 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:24:59.940613 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265caca3-325a-484e-9f02-2da31e17eb58 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:24:59.945308 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16606c3b-fdb1-40cf-962e-420a73528074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:24:59.945935 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16606c3b-fdb1-40cf-962e-420a73528074 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:24:59.946304 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16606c3b-fdb1-40cf-962e-420a73528074 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:24:59.946487 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16606c3b-fdb1-40cf-962e-420a73528074 admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:24:59.947161 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16606c3b-fdb1-40cf-962e-420a73528074 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:24:59.949121 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16606c3b-fdb1-40cf-962e-420a73528074 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:24:59.973770 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 776/1579] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:24:59 2022] POST /identity/v3/projects => generated 389 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:24:59.974797 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab50ac43-cb38-4d80-ae45-0cf58b46c483 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:00.035479 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:00.039162 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab50ac43-cb38-4d80-ae45-0cf58b46c483 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:00.044068 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5081b1-ad0c-4254-ba7e-f7301a5b01c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:00.044697 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5081b1-ad0c-4254-ba7e-f7301a5b01c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:00.044985 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5081b1-ad0c-4254-ba7e-f7301a5b01c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:00.045265 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5081b1-ad0c-4254-ba7e-f7301a5b01c3 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:00.045940 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5081b1-ad0c-4254-ba7e-f7301a5b01c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:00.047989 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5081b1-ad0c-4254-ba7e-f7301a5b01c3 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:00.059350 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 804/1580] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:24:59 2022] POST /identity/v3/users => generated 289 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:25:00.064149 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16606c3b-fdb1-40cf-962e-420a73528074 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:00.071041 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16606c3b-fdb1-40cf-962e-420a73528074 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:00.075913 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224d6793-7b1f-40f8-8a19-0f41d4e81c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:00.076539 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d6793-7b1f-40f8-8a19-0f41d4e81c1f admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:00.076820 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d6793-7b1f-40f8-8a19-0f41d4e81c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:00.077089 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d6793-7b1f-40f8-8a19-0f41d4e81c1f admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:00.077787 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224d6793-7b1f-40f8-8a19-0f41d4e81c1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:00.080244 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224d6793-7b1f-40f8-8a19-0f41d4e81c1f admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:00.182105 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 777/1581] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:24:59 2022] POST /identity/v3/users => generated 290 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:25:00.183042 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5081b1-ad0c-4254-ba7e-f7301a5b01c3 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:00.223604 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 805/1582] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:25:00 2022] POST /identity/v3/projects => generated 357 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:25:00.223604 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224d6793-7b1f-40f8-8a19-0f41d4e81c1f admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:00.257196 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5081b1-ad0c-4254-ba7e-f7301a5b01c3 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:00.284488 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:00.306698 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224d6793-7b1f-40f8-8a19-0f41d4e81c1f admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:00.311468 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0291dd05-e8e6-4180-a70f-0732a7a5283c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:00.312370 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0291dd05-e8e6-4180-a70f-0732a7a5283c admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:00.312668 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0291dd05-e8e6-4180-a70f-0732a7a5283c admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:00.312970 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0291dd05-e8e6-4180-a70f-0732a7a5283c admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:00.313687 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0291dd05-e8e6-4180-a70f-0732a7a5283c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:00.316274 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0291dd05-e8e6-4180-a70f-0732a7a5283c admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:00.320800 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c86917-d750-4604-8673-0f3f333bea3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:00.321561 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c86917-d750-4604-8673-0f3f333bea3b service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:00.321876 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c86917-d750-4604-8673-0f3f333bea3b service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:00.322123 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c86917-d750-4604-8673-0f3f333bea3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:00.335492 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c86917-d750-4604-8673-0f3f333bea3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:00.337007 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c86917-d750-4604-8673-0f3f333bea3b service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:00.354252 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 778/1583] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:25:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:25:00.355094 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c86917-d750-4604-8673-0f3f333bea3b service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:00.367475 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c86917-d750-4604-8673-0f3f333bea3b service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:00.372290 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c21cbb-47c3-42a5-bad4-28e9ec42e8ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:00.377421 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c21cbb-47c3-42a5-bad4-28e9ec42e8ed admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:00.377421 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c21cbb-47c3-42a5-bad4-28e9ec42e8ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:00.377421 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c21cbb-47c3-42a5-bad4-28e9ec42e8ed admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:00.377421 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c21cbb-47c3-42a5-bad4-28e9ec42e8ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:00.377421 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c21cbb-47c3-42a5-bad4-28e9ec42e8ed admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:00.423080 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 806/1584] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:25:00 2022] POST /identity/v3/users => generated 302 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:25:00.425308 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c28201-fbee-4c6f-9db0-a2fd16549183 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:00.425619 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c28201-fbee-4c6f-9db0-a2fd16549183 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:00.425895 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c28201-fbee-4c6f-9db0-a2fd16549183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:00.446209 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-66c28201-fbee-4c6f-9db0-a2fd16549183 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:25:00.447506 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-66c28201-fbee-4c6f-9db0-a2fd16549183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:00.455710 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 807/1585] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:25:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:25:00.458330 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c28201-fbee-4c6f-9db0-a2fd16549183 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:00.466558 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c28201-fbee-4c6f-9db0-a2fd16549183 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:00.471737 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d00cdf-52f6-46a3-bb35-14554117e616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:00.472358 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d00cdf-52f6-46a3-bb35-14554117e616 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:00.472664 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d00cdf-52f6-46a3-bb35-14554117e616 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:00.472936 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d00cdf-52f6-46a3-bb35-14554117e616 admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:00.473616 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d00cdf-52f6-46a3-bb35-14554117e616 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:00.475605 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d00cdf-52f6-46a3-bb35-14554117e616 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:00.489212 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 779/1586] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:25:00 2022] POST /identity/v3/users => generated 292 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:25:00.490324 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c21cbb-47c3-42a5-bad4-28e9ec42e8ed admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:00.503793 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c21cbb-47c3-42a5-bad4-28e9ec42e8ed admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:00.509494 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5387e1a7-6cd6-4256-af77-8ea82904e4bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:00.514807 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5387e1a7-6cd6-4256-af77-8ea82904e4bf admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:00.514807 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5387e1a7-6cd6-4256-af77-8ea82904e4bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:00.514807 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5387e1a7-6cd6-4256-af77-8ea82904e4bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:00.514807 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5387e1a7-6cd6-4256-af77-8ea82904e4bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:00.514807 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5387e1a7-6cd6-4256-af77-8ea82904e4bf admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:00.526983 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 780/1587] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:25:00 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:25:00.528662 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5387e1a7-6cd6-4256-af77-8ea82904e4bf admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:00.543451 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5387e1a7-6cd6-4256-af77-8ea82904e4bf admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:00.549566 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971cddff-550d-4bcd-a032-f9452fbe58b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:00.550202 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971cddff-550d-4bcd-a032-f9452fbe58b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:00.550485 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971cddff-550d-4bcd-a032-f9452fbe58b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:00.550760 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971cddff-550d-4bcd-a032-f9452fbe58b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:00.551629 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971cddff-550d-4bcd-a032-f9452fbe58b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:00.553286 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971cddff-550d-4bcd-a032-f9452fbe58b1 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:00.571892 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 781/1588] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:25:00 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:25:00.571892 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971cddff-550d-4bcd-a032-f9452fbe58b1 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:00.577476 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 808/1589] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:25:00 2022] POST /identity/v3/users => generated 295 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:25:00.578885 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d00cdf-52f6-46a3-bb35-14554117e616 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:00.582521 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971cddff-550d-4bcd-a032-f9452fbe58b1 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:00.589692 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d00cdf-52f6-46a3-bb35-14554117e616 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:00.589937 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1a42ad-6b91-4461-98e0-99eba9576017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:00.590563 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a42ad-6b91-4461-98e0-99eba9576017 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:00.590845 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a42ad-6b91-4461-98e0-99eba9576017 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:00.591118 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a42ad-6b91-4461-98e0-99eba9576017 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:00.591822 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1a42ad-6b91-4461-98e0-99eba9576017 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:00.593819 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1a42ad-6b91-4461-98e0-99eba9576017 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:00.595754 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914c0498-9aef-4fcf-90dc-7b03e7a2b497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:00.596408 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914c0498-9aef-4fcf-90dc-7b03e7a2b497 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:00.596928 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914c0498-9aef-4fcf-90dc-7b03e7a2b497 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:00.597208 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914c0498-9aef-4fcf-90dc-7b03e7a2b497 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:00.597923 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914c0498-9aef-4fcf-90dc-7b03e7a2b497 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:00.599639 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914c0498-9aef-4fcf-90dc-7b03e7a2b497 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:00.612822 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 809/1590] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:25:00 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:25:00.613831 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914c0498-9aef-4fcf-90dc-7b03e7a2b497 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:00.615680 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-914c0498-9aef-4fcf-90dc-7b03e7a2b497 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:00.628275 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914c0498-9aef-4fcf-90dc-7b03e7a2b497 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:00.632146 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c5352c-9df8-4d2b-a5f4-8236f3e9a5a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:00.632769 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c5352c-9df8-4d2b-a5f4-8236f3e9a5a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:00.633049 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c5352c-9df8-4d2b-a5f4-8236f3e9a5a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:00.633319 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c5352c-9df8-4d2b-a5f4-8236f3e9a5a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:00.643689 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c5352c-9df8-4d2b-a5f4-8236f3e9a5a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:00.645154 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c5352c-9df8-4d2b-a5f4-8236f3e9a5a7 service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:00.656358 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 810/1591] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:25:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:25:00.657801 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c5352c-9df8-4d2b-a5f4-8236f3e9a5a7 service neutron] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:00.667935 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c5352c-9df8-4d2b-a5f4-8236f3e9a5a7 service neutron] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:00.674863 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c109f25-d3e5-43e7-b212-962298adfb2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:00.675678 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c109f25-d3e5-43e7-b212-962298adfb2b admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:00.675810 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c109f25-d3e5-43e7-b212-962298adfb2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:00.676105 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c109f25-d3e5-43e7-b212-962298adfb2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:00.676820 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c109f25-d3e5-43e7-b212-962298adfb2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:00.678473 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c109f25-d3e5-43e7-b212-962298adfb2b admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:00.689968 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 782/1592] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:25:00 2022] POST /identity/v3/users => generated 279 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:25:00.690838 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1a42ad-6b91-4461-98e0-99eba9576017 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:00.692241 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 811/1593] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:25:00 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:25:00.693396 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c109f25-d3e5-43e7-b212-962298adfb2b admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:00.702462 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1a42ad-6b91-4461-98e0-99eba9576017 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:00.705707 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c109f25-d3e5-43e7-b212-962298adfb2b admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:00.708564 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62009ad-d852-45b4-bd99-a54da303a5e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:00.709215 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62009ad-d852-45b4-bd99-a54da303a5e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:00.709468 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62009ad-d852-45b4-bd99-a54da303a5e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:00.709738 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62009ad-d852-45b4-bd99-a54da303a5e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:00.710435 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62009ad-d852-45b4-bd99-a54da303a5e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:00.710569 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cc2905-d631-4ebf-9e69-d9a3791447da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:00.711124 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cc2905-d631-4ebf-9e69-d9a3791447da admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:00.711444 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cc2905-d631-4ebf-9e69-d9a3791447da admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:00.711722 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cc2905-d631-4ebf-9e69-d9a3791447da admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:00.712098 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62009ad-d852-45b4-bd99-a54da303a5e4 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:00.712422 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cc2905-d631-4ebf-9e69-d9a3791447da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:00.714211 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cc2905-d631-4ebf-9e69-d9a3791447da admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:00.726504 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 783/1594] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:25:00 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:25:00.727920 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62009ad-d852-45b4-bd99-a54da303a5e4 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:00.733106 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 812/1595] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:25:00 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:25:00.734328 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cc2905-d631-4ebf-9e69-d9a3791447da admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:00.739522 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62009ad-d852-45b4-bd99-a54da303a5e4 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:00.744638 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25950c3-0790-4c93-ac73-facb81a11d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:00.745268 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25950c3-0790-4c93-ac73-facb81a11d73 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:00.745578 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25950c3-0790-4c93-ac73-facb81a11d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:00.745726 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cc2905-d631-4ebf-9e69-d9a3791447da admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:00.745906 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25950c3-0790-4c93-ac73-facb81a11d73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:00.746764 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25950c3-0790-4c93-ac73-facb81a11d73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:00.749113 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25950c3-0790-4c93-ac73-facb81a11d73 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:00.751228 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441a9c9f-c55a-40e6-93f2-95277bb512ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:00.751960 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441a9c9f-c55a-40e6-93f2-95277bb512ad admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:00.752246 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441a9c9f-c55a-40e6-93f2-95277bb512ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:00.752517 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441a9c9f-c55a-40e6-93f2-95277bb512ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:00.753395 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441a9c9f-c55a-40e6-93f2-95277bb512ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:00.755077 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441a9c9f-c55a-40e6-93f2-95277bb512ad admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:00.761625 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 784/1596] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:25:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:25:00.766881 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25950c3-0790-4c93-ac73-facb81a11d73 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:00.774242 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 813/1597] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:25:00 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:25:00.775134 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441a9c9f-c55a-40e6-93f2-95277bb512ad admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:00.785070 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25950c3-0790-4c93-ac73-facb81a11d73 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:00.789391 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441a9c9f-c55a-40e6-93f2-95277bb512ad admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:00.791023 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46540106-5fdd-4aee-8a52-ac6b54352762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:00.792272 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46540106-5fdd-4aee-8a52-ac6b54352762 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:00.792556 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46540106-5fdd-4aee-8a52-ac6b54352762 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:00.792825 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46540106-5fdd-4aee-8a52-ac6b54352762 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:00.793637 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46540106-5fdd-4aee-8a52-ac6b54352762 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:00.795374 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46540106-5fdd-4aee-8a52-ac6b54352762 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:00.819640 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaba2bcd-befb-42ea-b851-276f3156b1f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:00.819640 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba2bcd-befb-42ea-b851-276f3156b1f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:00.819640 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba2bcd-befb-42ea-b851-276f3156b1f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:00.819640 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba2bcd-befb-42ea-b851-276f3156b1f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:00.820150 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaba2bcd-befb-42ea-b851-276f3156b1f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:00.820150 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaba2bcd-befb-42ea-b851-276f3156b1f4 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:00.820304 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 785/1598] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:25:00 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:25:00.820304 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46540106-5fdd-4aee-8a52-ac6b54352762 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:00.820471 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 814/1599] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:25:00 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:25:00.820545 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaba2bcd-befb-42ea-b851-276f3156b1f4 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:00.825352 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46540106-5fdd-4aee-8a52-ac6b54352762 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:00.831306 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b912cc00-610c-4a99-b2a3-7085dfc6e47c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:00.832020 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b912cc00-610c-4a99-b2a3-7085dfc6e47c admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:00.832301 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b912cc00-610c-4a99-b2a3-7085dfc6e47c admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:00.832577 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b912cc00-610c-4a99-b2a3-7085dfc6e47c admin admin] PATH_INFO: `/v3/projects/00814c2455be4608b2984adcf6353df2/users/9b29c3f2c29d4e2aba552c144cff94f8/roles/8feebc60363c4feb870718b1721e5324` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:00.834060 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaba2bcd-befb-42ea-b851-276f3156b1f4 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:00.840087 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6fd4b4-24d8-4fbb-9d89-2fe0c25224df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:00.841101 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6fd4b4-24d8-4fbb-9d89-2fe0c25224df admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:00.841404 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6fd4b4-24d8-4fbb-9d89-2fe0c25224df admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:00.841750 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6fd4b4-24d8-4fbb-9d89-2fe0c25224df admin admin] PATH_INFO: `/v3/projects/33e98e2129fe47bd8f81e1185d6a33a8/users/966b1056487d435ca83b6a9b15b0ecea/roles/8feebc60363c4feb870718b1721e5324` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:00.847308 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:00.855214 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b912cc00-610c-4a99-b2a3-7085dfc6e47c admin admin] RBAC: Authorizing `identity:create_grant(project_id=00814c2455be4608b2984adcf6353df2, user_id=9b29c3f2c29d4e2aba552c144cff94f8, role_id=8feebc60363c4feb870718b1721e5324)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:00.858464 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b912cc00-610c-4a99-b2a3-7085dfc6e47c admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:00.868972 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6fd4b4-24d8-4fbb-9d89-2fe0c25224df admin admin] RBAC: Authorizing `identity:create_grant(project_id=33e98e2129fe47bd8f81e1185d6a33a8, user_id=966b1056487d435ca83b6a9b15b0ecea, role_id=8feebc60363c4feb870718b1721e5324)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:00.871940 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6fd4b4-24d8-4fbb-9d89-2fe0c25224df admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:01.007186 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 786/1600] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:25:00 2022] PUT /identity/v3/projects/00814c2455be4608b2984adcf6353df2/users/9b29c3f2c29d4e2aba552c144cff94f8/roles/8feebc60363c4feb870718b1721e5324 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:25:01.007186 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b912cc00-610c-4a99-b2a3-7085dfc6e47c admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:01.009637 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 815/1601] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:25:00 2022] PUT /identity/v3/projects/33e98e2129fe47bd8f81e1185d6a33a8/users/966b1056487d435ca83b6a9b15b0ecea/roles/8feebc60363c4feb870718b1721e5324 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:25:01.011052 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6fd4b4-24d8-4fbb-9d89-2fe0c25224df admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:01.115145 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6fd4b4-24d8-4fbb-9d89-2fe0c25224df admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:01.121802 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09280cfb-ecfc-4730-8f9f-e95802d75f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:01.122501 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09280cfb-ecfc-4730-8f9f-e95802d75f7b admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:01.122789 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09280cfb-ecfc-4730-8f9f-e95802d75f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:01.123068 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09280cfb-ecfc-4730-8f9f-e95802d75f7b admin admin] PATH_INFO: `/v3/projects/a1672ce405d74c32919c98f4ae3d4c65/users/80edbdb9f17d478e943729bdde2ca036/roles/8feebc60363c4feb870718b1721e5324` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:01.132852 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b912cc00-610c-4a99-b2a3-7085dfc6e47c admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:01.140618 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6ea672-bdf2-47bd-b880-dfd06b55d2b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:01.141018 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09280cfb-ecfc-4730-8f9f-e95802d75f7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1672ce405d74c32919c98f4ae3d4c65, user_id=80edbdb9f17d478e943729bdde2ca036, role_id=8feebc60363c4feb870718b1721e5324)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:01.141486 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6ea672-bdf2-47bd-b880-dfd06b55d2b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:01.141775 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6ea672-bdf2-47bd-b880-dfd06b55d2b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:01.142052 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6ea672-bdf2-47bd-b880-dfd06b55d2b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:01.143433 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6ea672-bdf2-47bd-b880-dfd06b55d2b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:01.145235 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09280cfb-ecfc-4730-8f9f-e95802d75f7b admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:01.145416 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6ea672-bdf2-47bd-b880-dfd06b55d2b0 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:01.149232 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:01.163034 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 787/1602] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:25:01 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:25:01.163034 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6ea672-bdf2-47bd-b880-dfd06b55d2b0 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:01.175746 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6ea672-bdf2-47bd-b880-dfd06b55d2b0 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:01.184061 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa569ed-e206-479f-8345-c3d778d15b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:01.184238 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa569ed-e206-479f-8345-c3d778d15b7c admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:01.184447 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa569ed-e206-479f-8345-c3d778d15b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:01.184710 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa569ed-e206-479f-8345-c3d778d15b7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:01.185960 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa569ed-e206-479f-8345-c3d778d15b7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:01.188449 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa569ed-e206-479f-8345-c3d778d15b7c admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:01.202126 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 788/1603] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:25:01 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:25:01.203452 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa569ed-e206-479f-8345-c3d778d15b7c admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:01.214936 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa569ed-e206-479f-8345-c3d778d15b7c admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:01.221477 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2129212-6187-4853-9d59-8f2871a79f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:01.222394 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2129212-6187-4853-9d59-8f2871a79f06 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:01.222754 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2129212-6187-4853-9d59-8f2871a79f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:01.223107 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2129212-6187-4853-9d59-8f2871a79f06 admin admin] PATH_INFO: `/v3/projects/9c97e87ed5cc4cd5b928e4b8e5be8053/users/f44c8dee7121485b81d22764abe990bc/roles/8feebc60363c4feb870718b1721e5324` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:01.242655 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2129212-6187-4853-9d59-8f2871a79f06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c97e87ed5cc4cd5b928e4b8e5be8053, user_id=f44c8dee7121485b81d22764abe990bc, role_id=8feebc60363c4feb870718b1721e5324)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:01.245573 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2129212-6187-4853-9d59-8f2871a79f06 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:01.295619 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 816/1604] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:25:01 2022] PUT /identity/v3/projects/a1672ce405d74c32919c98f4ae3d4c65/users/80edbdb9f17d478e943729bdde2ca036/roles/8feebc60363c4feb870718b1721e5324 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:25:01.301669 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a579408-e4eb-4d79-b3ca-8b70126f61bc None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:01.301983 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a579408-e4eb-4d79-b3ca-8b70126f61bc None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:01.302736 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a579408-e4eb-4d79-b3ca-8b70126f61bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:01.325405 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-9a579408-e4eb-4d79-b3ca-8b70126f61bc None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:25:01.326729 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-9a579408-e4eb-4d79-b3ca-8b70126f61bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:01.347499 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 789/1605] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:25:01 2022] PUT /identity/v3/projects/9c97e87ed5cc4cd5b928e4b8e5be8053/users/f44c8dee7121485b81d22764abe990bc/roles/8feebc60363c4feb870718b1721e5324 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:25:01.359136 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8905dece-7819-4ee3-b12c-73999c14f0ce None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:01.359136 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8905dece-7819-4ee3-b12c-73999c14f0ce None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:01.359136 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8905dece-7819-4ee3-b12c-73999c14f0ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:01.384371 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-8905dece-7819-4ee3-b12c-73999c14f0ce None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:25:01.385894 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-8905dece-7819-4ee3-b12c-73999c14f0ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:01.437843 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 817/1606] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:25:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:25:01.437843 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a579408-e4eb-4d79-b3ca-8b70126f61bc None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:01.463058 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a579408-e4eb-4d79-b3ca-8b70126f61bc None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:01.500094 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 790/1607] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:25:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:25:01.502166 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8905dece-7819-4ee3-b12c-73999c14f0ce None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:01.522635 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8905dece-7819-4ee3-b12c-73999c14f0ce None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:01.529069 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff10980-2b39-4ef0-91ad-677bd228b79f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:01.529575 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff10980-2b39-4ef0-91ad-677bd228b79f admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:01.530078 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff10980-2b39-4ef0-91ad-677bd228b79f admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:01.530590 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff10980-2b39-4ef0-91ad-677bd228b79f admin admin] PATH_INFO: `/v3/projects/ecb46496afcd4f3fb0f689c172a6603e/users/a92fe8a407de490fb24d4870c234b43d/roles/8feebc60363c4feb870718b1721e5324` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:01.540305 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b647dc2d-1a68-4449-acec-dfd24536fa33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:01.543832 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b647dc2d-1a68-4449-acec-dfd24536fa33 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:01.543832 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b647dc2d-1a68-4449-acec-dfd24536fa33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:01.543832 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b647dc2d-1a68-4449-acec-dfd24536fa33 admin admin] PATH_INFO: `/v3/projects/48234ebc0b1341719ebf2a9419cb6b54/users/4087b29e666c4afaa36118e82eb169d3/roles/8feebc60363c4feb870718b1721e5324` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:01.557958 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff10980-2b39-4ef0-91ad-677bd228b79f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecb46496afcd4f3fb0f689c172a6603e, user_id=a92fe8a407de490fb24d4870c234b43d, role_id=8feebc60363c4feb870718b1721e5324)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:01.561365 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff10980-2b39-4ef0-91ad-677bd228b79f admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:01.565337 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b647dc2d-1a68-4449-acec-dfd24536fa33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48234ebc0b1341719ebf2a9419cb6b54, user_id=4087b29e666c4afaa36118e82eb169d3, role_id=8feebc60363c4feb870718b1721e5324)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:01.570224 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b647dc2d-1a68-4449-acec-dfd24536fa33 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:01.686003 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 791/1608] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:25:01 2022] PUT /identity/v3/projects/ecb46496afcd4f3fb0f689c172a6603e/users/a92fe8a407de490fb24d4870c234b43d/roles/8feebc60363c4feb870718b1721e5324 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:25:01.699078 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be27f853-3ba5-499e-8488-5106c01009c4 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:01.699416 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be27f853-3ba5-499e-8488-5106c01009c4 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:01.699705 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be27f853-3ba5-499e-8488-5106c01009c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:01.720056 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-be27f853-3ba5-499e-8488-5106c01009c4 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:25:01.721479 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-be27f853-3ba5-499e-8488-5106c01009c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:01.746919 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 818/1609] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:25:01 2022] PUT /identity/v3/projects/48234ebc0b1341719ebf2a9419cb6b54/users/4087b29e666c4afaa36118e82eb169d3/roles/8feebc60363c4feb870718b1721e5324 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:25:01.748208 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8630ca-f5dd-4cfe-81a4-420377f86453 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:01.748856 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8630ca-f5dd-4cfe-81a4-420377f86453 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:01.749249 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8630ca-f5dd-4cfe-81a4-420377f86453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:01.758258 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:01.786090 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-dd8630ca-f5dd-4cfe-81a4-420377f86453 None None] MFA Rules not processed for user `f44c8dee7121485b81d22764abe990bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:25:01.787914 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-dd8630ca-f5dd-4cfe-81a4-420377f86453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:01.811750 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 792/1610] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:25:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:25:01.812886 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be27f853-3ba5-499e-8488-5106c01009c4 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:01.898407 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be27f853-3ba5-499e-8488-5106c01009c4 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:01.927194 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 819/1611] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:25:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:25:01.928985 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8630ca-f5dd-4cfe-81a4-420377f86453 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:01.941508 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:01.980403 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fd6ca5-5e85-4004-a52b-27ab760e50a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:01.981074 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd6ca5-5e85-4004-a52b-27ab760e50a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:01.981350 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd6ca5-5e85-4004-a52b-27ab760e50a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:01.981626 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd6ca5-5e85-4004-a52b-27ab760e50a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:02.026119 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8630ca-f5dd-4cfe-81a4-420377f86453 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:02.036112 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca55c8f-e121-49b2-9fd7-87bb5f12fd52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:02.036900 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca55c8f-e121-49b2-9fd7-87bb5f12fd52 service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:02.037433 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca55c8f-e121-49b2-9fd7-87bb5f12fd52 service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:02.037795 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca55c8f-e121-49b2-9fd7-87bb5f12fd52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:02.076689 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fd6ca5-5e85-4004-a52b-27ab760e50a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:02.077766 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fd6ca5-5e85-4004-a52b-27ab760e50a2 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:02.091373 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 793/1612] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:25:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:25:02.092978 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b04f0c4-ae72-47e7-ad5e-9de93151b547 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:02.093706 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b04f0c4-ae72-47e7-ad5e-9de93151b547 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:02.093706 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b04f0c4-ae72-47e7-ad5e-9de93151b547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:02.109216 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca55c8f-e121-49b2-9fd7-87bb5f12fd52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:02.111517 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca55c8f-e121-49b2-9fd7-87bb5f12fd52 service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:02.123283 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 820/1613] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:25:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:25:02.124706 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c443d-0a9c-44f7-8a15-0e6a00433911 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:02.125001 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c443d-0a9c-44f7-8a15-0e6a00433911 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:02.125284 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c443d-0a9c-44f7-8a15-0e6a00433911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:02.130653 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-4b04f0c4-ae72-47e7-ad5e-9de93151b547 None None] MFA Rules not processed for user `a92fe8a407de490fb24d4870c234b43d`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:25:02.132181 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-4b04f0c4-ae72-47e7-ad5e-9de93151b547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:02.152260 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-e31c443d-0a9c-44f7-8a15-0e6a00433911 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:25:02.153813 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-e31c443d-0a9c-44f7-8a15-0e6a00433911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:02.164149 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 821/1614] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:25:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:25:02.164728 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31c443d-0a9c-44f7-8a15-0e6a00433911 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:02.178624 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:02.181541 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31c443d-0a9c-44f7-8a15-0e6a00433911 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:02.189290 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aaab8c-306d-4e22-bf4f-7a651a34c5ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:02.196063 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aaab8c-306d-4e22-bf4f-7a651a34c5ba service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:02.196340 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aaab8c-306d-4e22-bf4f-7a651a34c5ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:02.196630 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aaab8c-306d-4e22-bf4f-7a651a34c5ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:02.216544 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aaab8c-306d-4e22-bf4f-7a651a34c5ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:02.229140 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aaab8c-306d-4e22-bf4f-7a651a34c5ba service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:02.248878 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 794/1615] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:25:02 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:25:02.248878 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34759e4-c652-4944-bcc2-09ab13e5788a None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:02.248878 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34759e4-c652-4944-bcc2-09ab13e5788a None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:02.248878 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34759e4-c652-4944-bcc2-09ab13e5788a None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:02.255774 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 795/1616] 10.222.0.25 () {54 vars in 962 bytes} [Thu Aug 4 21:25:02 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:25:02.257638 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34759e4-c652-4944-bcc2-09ab13e5788a None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:02.257809 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 822/1617] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:25:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:25:02.258693 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aaab8c-306d-4e22-bf4f-7a651a34c5ba service neutron] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:02.270260 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aaab8c-306d-4e22-bf4f-7a651a34c5ba service neutron] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:02.277371 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d476eb71-0db2-4e3f-a6b9-2f78f09bb830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:02.278156 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d476eb71-0db2-4e3f-a6b9-2f78f09bb830 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:02.278431 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d476eb71-0db2-4e3f-a6b9-2f78f09bb830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:02.278710 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d476eb71-0db2-4e3f-a6b9-2f78f09bb830 admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:02.279435 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d476eb71-0db2-4e3f-a6b9-2f78f09bb830 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:02.281966 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d476eb71-0db2-4e3f-a6b9-2f78f09bb830 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:02.283927 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34759e4-c652-4944-bcc2-09ab13e5788a None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:02.289123 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e19846-fdc3-474d-947d-9405a02e4b9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:02.289969 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e19846-fdc3-474d-947d-9405a02e4b9a service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:02.290407 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e19846-fdc3-474d-947d-9405a02e4b9a service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:02.290825 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e19846-fdc3-474d-947d-9405a02e4b9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:02.305155 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e19846-fdc3-474d-947d-9405a02e4b9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:02.306985 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e19846-fdc3-474d-947d-9405a02e4b9a service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:02.318602 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 796/1618] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:25:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:25:02.319955 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aeaf63-dfa2-4ce1-bfe2-58442adeb5b2 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:02.320079 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aeaf63-dfa2-4ce1-bfe2-58442adeb5b2 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:02.320286 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aeaf63-dfa2-4ce1-bfe2-58442adeb5b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:02.389837 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-55aeaf63-dfa2-4ce1-bfe2-58442adeb5b2 None None] MFA Rules not processed for user `6a2ce1df87fa47e5be301916bf3245fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:25:02.392811 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-55aeaf63-dfa2-4ce1-bfe2-58442adeb5b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:02.429263 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 823/1619] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:25:02 2022] POST /identity/v3/projects => generated 357 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:25:02.433760 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d476eb71-0db2-4e3f-a6b9-2f78f09bb830 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:02.536306 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 797/1620] 10.222.0.25 () {58 vars in 1059 bytes} [Thu Aug 4 21:25:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:25:02.545034 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aeaf63-dfa2-4ce1-bfe2-58442adeb5b2 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:02.562747 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d476eb71-0db2-4e3f-a6b9-2f78f09bb830 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:02.571483 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ede41b0-8b72-45a2-8837-7b114c05b34a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:02.572136 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ede41b0-8b72-45a2-8837-7b114c05b34a admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:02.572478 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ede41b0-8b72-45a2-8837-7b114c05b34a admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:02.572838 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ede41b0-8b72-45a2-8837-7b114c05b34a admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:02.573079 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aeaf63-dfa2-4ce1-bfe2-58442adeb5b2 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:02.573613 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ede41b0-8b72-45a2-8837-7b114c05b34a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:02.576282 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ede41b0-8b72-45a2-8837-7b114c05b34a admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:02.692948 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024ffda0-268d-4635-9f17-b227c162a95a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a2ce1df87fa47e5be301916bf3245fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:02.693899 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ffda0-268d-4635-9f17-b227c162a95a service nova] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:02.694287 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ffda0-268d-4635-9f17-b227c162a95a service nova] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:02.695205 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ffda0-268d-4635-9f17-b227c162a95a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:02.709221 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 824/1621] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:25:02 2022] POST /identity/v3/users => generated 285 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:25:02.710738 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ede41b0-8b72-45a2-8837-7b114c05b34a admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:02.727147 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ede41b0-8b72-45a2-8837-7b114c05b34a admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:02.741138 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:02.744616 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7e7ea5-5a72-48ad-87cf-87da46f221c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:02.745443 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7e7ea5-5a72-48ad-87cf-87da46f221c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:02.745727 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7e7ea5-5a72-48ad-87cf-87da46f221c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:02.751078 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7e7ea5-5a72-48ad-87cf-87da46f221c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:02.751078 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7e7ea5-5a72-48ad-87cf-87da46f221c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:02.751078 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7e7ea5-5a72-48ad-87cf-87da46f221c2 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:02.772098 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 825/1622] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:25:02 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:25:02.776618 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7e7ea5-5a72-48ad-87cf-87da46f221c2 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:02.800738 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7e7ea5-5a72-48ad-87cf-87da46f221c2 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:02.807156 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024ffda0-268d-4635-9f17-b227c162a95a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:02.810974 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024ffda0-268d-4635-9f17-b227c162a95a service nova] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:02.812531 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6986ed4-1893-456e-9ba6-094d3b4dbb1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:02.813238 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6986ed4-1893-456e-9ba6-094d3b4dbb1b admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:02.813524 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6986ed4-1893-456e-9ba6-094d3b4dbb1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:02.813806 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6986ed4-1893-456e-9ba6-094d3b4dbb1b admin admin] PATH_INFO: `/v3/projects/51b088994e4b40c5b54627cfbf7ae353/users/4bedebd0e83147e4922a4f85bc6e3536/roles/a432dd4735944ecfab2fa5ce1705d356` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:02.827555 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 798/1623] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:25:02 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:25:02.865050 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6986ed4-1893-456e-9ba6-094d3b4dbb1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=51b088994e4b40c5b54627cfbf7ae353, user_id=4bedebd0e83147e4922a4f85bc6e3536, role_id=a432dd4735944ecfab2fa5ce1705d356)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:02.867643 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6986ed4-1893-456e-9ba6-094d3b4dbb1b admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:03.021384 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:03.027391 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 826/1624] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:25:02 2022] PUT /identity/v3/projects/51b088994e4b40c5b54627cfbf7ae353/users/4bedebd0e83147e4922a4f85bc6e3536/roles/a432dd4735944ecfab2fa5ce1705d356 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:25:03.032387 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024ffda0-268d-4635-9f17-b227c162a95a service nova] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:03.149480 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024ffda0-268d-4635-9f17-b227c162a95a service nova] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:03.150360 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15f4084-d91f-4461-b163-efa237ccb510 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:03.150360 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15f4084-d91f-4461-b163-efa237ccb510 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:03.150360 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15f4084-d91f-4461-b163-efa237ccb510 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:03.150360 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 827/1625] 10.222.0.25 () {54 vars in 912 bytes} [Thu Aug 4 21:25:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 21:25:03.150360 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf3133-700b-4f51-87bd-80c5edcf6abb None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:03.150360 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf3133-700b-4f51-87bd-80c5edcf6abb None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:03.150875 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf3133-700b-4f51-87bd-80c5edcf6abb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:03.166464 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-efaf3133-700b-4f51-87bd-80c5edcf6abb None None] MFA Rules not processed for user `6a2ce1df87fa47e5be301916bf3245fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:25:03.167927 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-efaf3133-700b-4f51-87bd-80c5edcf6abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:03.269816 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 828/1626] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Aug 4 21:25:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:25:03.269816 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaf3133-700b-4f51-87bd-80c5edcf6abb None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:03.288857 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d868fe90-c3a1-4dbd-a250-d0f048a74c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:03.289720 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d868fe90-c3a1-4dbd-a250-d0f048a74c6a admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:03.290693 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d868fe90-c3a1-4dbd-a250-d0f048a74c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:03.290981 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d868fe90-c3a1-4dbd-a250-d0f048a74c6a admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:03.291850 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d868fe90-c3a1-4dbd-a250-d0f048a74c6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:03.294891 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d868fe90-c3a1-4dbd-a250-d0f048a74c6a admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:03.326923 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 799/1627] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:25:03 2022] GET /identity/v3/roles => generated 1420 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:25:03.331198 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d868fe90-c3a1-4dbd-a250-d0f048a74c6a admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:03.346675 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d868fe90-c3a1-4dbd-a250-d0f048a74c6a admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:03.353878 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a74070-1647-4810-ba1f-a6c9cec1eca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:03.355050 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a74070-1647-4810-ba1f-a6c9cec1eca1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:03.355664 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a74070-1647-4810-ba1f-a6c9cec1eca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:03.356209 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a74070-1647-4810-ba1f-a6c9cec1eca1 admin admin] PATH_INFO: `/v3/projects/51b088994e4b40c5b54627cfbf7ae353/users/4bedebd0e83147e4922a4f85bc6e3536/roles/a432dd4735944ecfab2fa5ce1705d356` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:03.359140 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a74070-1647-4810-ba1f-a6c9cec1eca1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51b088994e4b40c5b54627cfbf7ae353, user_id=4bedebd0e83147e4922a4f85bc6e3536, role_id=a432dd4735944ecfab2fa5ce1705d356)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:03.363974 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a74070-1647-4810-ba1f-a6c9cec1eca1 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:03.395014 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 800/1628] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:25:03 2022] PUT /identity/v3/projects/51b088994e4b40c5b54627cfbf7ae353/users/4bedebd0e83147e4922a4f85bc6e3536/roles/a432dd4735944ecfab2fa5ce1705d356 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:25:03.409554 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaf3133-700b-4f51-87bd-80c5edcf6abb None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:03.414419 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e3b15-fa83-4ac8-9e21-8d1a4d99bd89 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:03.415700 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e3b15-fa83-4ac8-9e21-8d1a4d99bd89 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:03.416541 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e3b15-fa83-4ac8-9e21-8d1a4d99bd89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:03.484817 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-414e3b15-fa83-4ac8-9e21-8d1a4d99bd89 None None] MFA Rules not processed for user `4bedebd0e83147e4922a4f85bc6e3536`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:25:03.489216 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-414e3b15-fa83-4ac8-9e21-8d1a4d99bd89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:03.518145 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d873ad21-167d-45be-b1bc-e426f6c27de3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e31f2728056b460fa474478eb85ca1d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:03.518896 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d873ad21-167d-45be-b1bc-e426f6c27de3 service cinder] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:03.519116 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d873ad21-167d-45be-b1bc-e426f6c27de3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:03.519393 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d873ad21-167d-45be-b1bc-e426f6c27de3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:03.655687 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d873ad21-167d-45be-b1bc-e426f6c27de3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:03.659952 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d873ad21-167d-45be-b1bc-e426f6c27de3 service cinder] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:04.090353 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 829/1629] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:25:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:25:04.095000 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 801/1630] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:25:03 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:25:04.099126 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d873ad21-167d-45be-b1bc-e426f6c27de3 service cinder] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:04.099126 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d873ad21-167d-45be-b1bc-e426f6c27de3 service cinder] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:04.099126 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6d2921-489c-4878-9ff1-94847c6bcb25 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e31f2728056b460fa474478eb85ca1d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:04.099126 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6d2921-489c-4878-9ff1-94847c6bcb25 service cinder] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:04.101799 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6d2921-489c-4878-9ff1-94847c6bcb25 service cinder] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:04.101799 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6d2921-489c-4878-9ff1-94847c6bcb25 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:04.101799 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6d2921-489c-4878-9ff1-94847c6bcb25 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:04.101799 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6d2921-489c-4878-9ff1-94847c6bcb25 service cinder] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:04.101799 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 830/1631] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:25:03 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:25:04.101799 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:04.337990 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414e3b15-fa83-4ac8-9e21-8d1a4d99bd89 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:04.357561 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414e3b15-fa83-4ac8-9e21-8d1a4d99bd89 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:04.369253 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec183cec-11a0-4eed-b248-e51e21a7ac4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e31f2728056b460fa474478eb85ca1d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:04.385125 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec183cec-11a0-4eed-b248-e51e21a7ac4a service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:04.385125 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec183cec-11a0-4eed-b248-e51e21a7ac4a service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:04.385125 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec183cec-11a0-4eed-b248-e51e21a7ac4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:04.446668 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:04.512723 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec183cec-11a0-4eed-b248-e51e21a7ac4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:04.514227 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec183cec-11a0-4eed-b248-e51e21a7ac4a service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:04.534151 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 802/1632] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:25:04 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:25:06.047132 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa9a4ac-a33c-4f52-a5dd-8aa30de26a49 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:06.047784 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa9a4ac-a33c-4f52-a5dd-8aa30de26a49 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:06.047784 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa9a4ac-a33c-4f52-a5dd-8aa30de26a49 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:06.050170 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 831/1633] 10.222.0.25 () {54 vars in 962 bytes} [Thu Aug 4 21:25:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:25:06.058109 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e211d1-a01d-4aea-bc50-6e1d54d5ff72 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:06.058668 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e211d1-a01d-4aea-bc50-6e1d54d5ff72 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:06.058668 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e211d1-a01d-4aea-bc50-6e1d54d5ff72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:06.089151 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-59e211d1-a01d-4aea-bc50-6e1d54d5ff72 None None] MFA Rules not processed for user `6a2ce1df87fa47e5be301916bf3245fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:25:06.091020 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-59e211d1-a01d-4aea-bc50-6e1d54d5ff72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:06.111804 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89c0653-92c6-4969-a404-77b465b09af7 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:06.112609 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89c0653-92c6-4969-a404-77b465b09af7 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:06.112609 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89c0653-92c6-4969-a404-77b465b09af7 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:06.114514 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 832/1634] 10.222.0.25 () {54 vars in 912 bytes} [Thu Aug 4 21:25:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:25:06.121926 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68cfbc3-4e14-4e5e-b3df-bda81cd7aa63 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:06.122303 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68cfbc3-4e14-4e5e-b3df-bda81cd7aa63 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:06.122576 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68cfbc3-4e14-4e5e-b3df-bda81cd7aa63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:06.133082 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 803/1635] 10.222.0.25 () {58 vars in 1059 bytes} [Thu Aug 4 21:25:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:25:06.165200 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-b68cfbc3-4e14-4e5e-b3df-bda81cd7aa63 None None] MFA Rules not processed for user `6a2ce1df87fa47e5be301916bf3245fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:25:06.166857 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-b68cfbc3-4e14-4e5e-b3df-bda81cd7aa63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:06.210551 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 833/1636] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Aug 4 21:25:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:25:06.234639 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e211d1-a01d-4aea-bc50-6e1d54d5ff72 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:06.274759 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e211d1-a01d-4aea-bc50-6e1d54d5ff72 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:06.290451 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267b1ba4-ada0-4c5a-910f-c7298bf16032 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e31f2728056b460fa474478eb85ca1d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:06.291414 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267b1ba4-ada0-4c5a-910f-c7298bf16032 service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:06.291863 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267b1ba4-ada0-4c5a-910f-c7298bf16032 service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:06.292271 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267b1ba4-ada0-4c5a-910f-c7298bf16032 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:06.331869 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267b1ba4-ada0-4c5a-910f-c7298bf16032 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:06.333770 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267b1ba4-ada0-4c5a-910f-c7298bf16032 service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:06.384716 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 804/1637] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:25:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:25:06.811700 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6d3b9a-2e28-425b-9ad4-f8a7dba68b3a None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:06.812260 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6d3b9a-2e28-425b-9ad4-f8a7dba68b3a None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:06.812537 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6d3b9a-2e28-425b-9ad4-f8a7dba68b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:06.870438 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-ee6d3b9a-2e28-425b-9ad4-f8a7dba68b3a None None] MFA Rules not processed for user `80edbdb9f17d478e943729bdde2ca036`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:25:06.871777 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-ee6d3b9a-2e28-425b-9ad4-f8a7dba68b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:07.061437 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267b1ba4-ada0-4c5a-910f-c7298bf16032 service cinder] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:07.062234 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 834/1638] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:25:06 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:25:07.197202 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267b1ba4-ada0-4c5a-910f-c7298bf16032 service cinder] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:07.220766 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470a588a-d691-43dd-ba41-404e958c9b69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:07.223114 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470a588a-d691-43dd-ba41-404e958c9b69 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:07.224628 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470a588a-d691-43dd-ba41-404e958c9b69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:07.225055 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470a588a-d691-43dd-ba41-404e958c9b69 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:07.227177 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470a588a-d691-43dd-ba41-404e958c9b69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:07.233810 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470a588a-d691-43dd-ba41-404e958c9b69 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:07.381476 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 805/1639] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:25:07 2022] POST /identity/v3/projects => generated 403 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:25:07.383102 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6d3b9a-2e28-425b-9ad4-f8a7dba68b3a None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:07.395560 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6d3b9a-2e28-425b-9ad4-f8a7dba68b3a None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:07.507480 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675e5386-044f-4cb4-9932-b3baa8e89755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:07.508468 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675e5386-044f-4cb4-9932-b3baa8e89755 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:07.508916 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675e5386-044f-4cb4-9932-b3baa8e89755 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:07.509352 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675e5386-044f-4cb4-9932-b3baa8e89755 admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:07.510246 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675e5386-044f-4cb4-9932-b3baa8e89755 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:07.513792 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675e5386-044f-4cb4-9932-b3baa8e89755 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:07.703579 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 835/1640] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:25:07 2022] POST /identity/v3/users => generated 308 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:25:07.707900 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470a588a-d691-43dd-ba41-404e958c9b69 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:07.734300 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470a588a-d691-43dd-ba41-404e958c9b69 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:07.750101 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3fc035-5da1-4e82-8132-e899e1e92f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:07.753309 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3fc035-5da1-4e82-8132-e899e1e92f44 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:07.756098 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3fc035-5da1-4e82-8132-e899e1e92f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:07.756563 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3fc035-5da1-4e82-8132-e899e1e92f44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:07.760279 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3fc035-5da1-4e82-8132-e899e1e92f44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:07.765100 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3fc035-5da1-4e82-8132-e899e1e92f44 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:07.797807 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 806/1641] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:25:07 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:25:07.798087 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675e5386-044f-4cb4-9932-b3baa8e89755 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:07.813837 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675e5386-044f-4cb4-9932-b3baa8e89755 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:07.841226 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0221f020-a56a-4a69-b5ef-0d2b661c52c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:07.844176 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0221f020-a56a-4a69-b5ef-0d2b661c52c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:07.845811 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0221f020-a56a-4a69-b5ef-0d2b661c52c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:07.846156 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0221f020-a56a-4a69-b5ef-0d2b661c52c4 admin admin] PATH_INFO: `/v3/projects/bc03a87905644bf68b2d3fcef84cadac/users/4488f60b29364a02a1c2423e1c8c445e/roles/a432dd4735944ecfab2fa5ce1705d356` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:07.908330 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0221f020-a56a-4a69-b5ef-0d2b661c52c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc03a87905644bf68b2d3fcef84cadac, user_id=4488f60b29364a02a1c2423e1c8c445e, role_id=a432dd4735944ecfab2fa5ce1705d356)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:07.913378 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0221f020-a56a-4a69-b5ef-0d2b661c52c4 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:07.920844 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:08.052445 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 836/1642] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:25:07 2022] PUT /identity/v3/projects/bc03a87905644bf68b2d3fcef84cadac/users/4488f60b29364a02a1c2423e1c8c445e/roles/a432dd4735944ecfab2fa5ce1705d356 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:25:08.057425 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3fc035-5da1-4e82-8132-e899e1e92f44 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:08.083291 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3fc035-5da1-4e82-8132-e899e1e92f44 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:08.270761 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ea1b92-adaf-4cd5-b963-2562f84e36d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:08.271920 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ea1b92-adaf-4cd5-b963-2562f84e36d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:08.272888 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ea1b92-adaf-4cd5-b963-2562f84e36d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:08.273840 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ea1b92-adaf-4cd5-b963-2562f84e36d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:08.275866 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ea1b92-adaf-4cd5-b963-2562f84e36d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:08.281679 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ea1b92-adaf-4cd5-b963-2562f84e36d8 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:08.328344 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 807/1643] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:25:08 2022] GET /identity/v3/roles => generated 1420 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:25:08.332473 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0221f020-a56a-4a69-b5ef-0d2b661c52c4 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:08.357608 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0221f020-a56a-4a69-b5ef-0d2b661c52c4 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:08.368660 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af093e6-39fb-4ed0-a04f-5c346698d283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:08.370109 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af093e6-39fb-4ed0-a04f-5c346698d283 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:08.371677 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af093e6-39fb-4ed0-a04f-5c346698d283 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:08.372070 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af093e6-39fb-4ed0-a04f-5c346698d283 admin admin] PATH_INFO: `/v3/projects/bc03a87905644bf68b2d3fcef84cadac/users/4488f60b29364a02a1c2423e1c8c445e/roles/a432dd4735944ecfab2fa5ce1705d356` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:08.377876 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af093e6-39fb-4ed0-a04f-5c346698d283 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc03a87905644bf68b2d3fcef84cadac, user_id=4488f60b29364a02a1c2423e1c8c445e, role_id=a432dd4735944ecfab2fa5ce1705d356)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:08.382548 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af093e6-39fb-4ed0-a04f-5c346698d283 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:08.469131 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 837/1644] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:25:08 2022] PUT /identity/v3/projects/bc03a87905644bf68b2d3fcef84cadac/users/4488f60b29364a02a1c2423e1c8c445e/roles/a432dd4735944ecfab2fa5ce1705d356 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:25:12.840378 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c87ed1-63cc-49ad-9b89-4c5cd99d3f8e None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:12.840378 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c87ed1-63cc-49ad-9b89-4c5cd99d3f8e None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:12.840378 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c87ed1-63cc-49ad-9b89-4c5cd99d3f8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:12.941948 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-31c87ed1-63cc-49ad-9b89-4c5cd99d3f8e None None] MFA Rules not processed for user `4488f60b29364a02a1c2423e1c8c445e`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:25:12.945711 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-31c87ed1-63cc-49ad-9b89-4c5cd99d3f8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:13.002693 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:13.176356 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 808/1645] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:25:12 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:25:14.709364 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7f9795-8d21-46d3-a6b1-f4d6b90bd289 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:14.711001 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7f9795-8d21-46d3-a6b1-f4d6b90bd289 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:14.711001 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7f9795-8d21-46d3-a6b1-f4d6b90bd289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:14.757648 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-ab7f9795-8d21-46d3-a6b1-f4d6b90bd289 None None] MFA Rules not processed for user `3d5f53509b2f42269cd26e77c79ad8be`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:25:14.759127 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-ab7f9795-8d21-46d3-a6b1-f4d6b90bd289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:14.972824 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368e3fd7-ddea-4145-b9ea-dc93f1dc6a93 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:14.972824 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368e3fd7-ddea-4145-b9ea-dc93f1dc6a93 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:14.972824 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368e3fd7-ddea-4145-b9ea-dc93f1dc6a93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:14.982087 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 838/1646] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:25:14 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:25:15.028375 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-368e3fd7-ddea-4145-b9ea-dc93f1dc6a93 None None] MFA Rules not processed for user `3d5f53509b2f42269cd26e77c79ad8be`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:25:15.029849 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-368e3fd7-ddea-4145-b9ea-dc93f1dc6a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:15.059117 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 809/1647] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:25:14 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:25:15.087813 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7f9795-8d21-46d3-a6b1-f4d6b90bd289 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:15.273332 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7f9795-8d21-46d3-a6b1-f4d6b90bd289 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:15.290807 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e6042e-831e-47e0-b460-830d462f871a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a2ce1df87fa47e5be301916bf3245fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:15.293183 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e6042e-831e-47e0-b460-830d462f871a service nova] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:15.299713 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e6042e-831e-47e0-b460-830d462f871a service nova] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:15.299713 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e6042e-831e-47e0-b460-830d462f871a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:15.313508 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e6042e-831e-47e0-b460-830d462f871a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:15.316233 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e6042e-831e-47e0-b460-830d462f871a service nova] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:15.332961 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 839/1648] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:25:15 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:25:15.718962 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368e3fd7-ddea-4145-b9ea-dc93f1dc6a93 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:15.900420 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368e3fd7-ddea-4145-b9ea-dc93f1dc6a93 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:15.913154 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652c6562-4a20-4e4f-8ac5-43c47e85e90e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:15.914288 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652c6562-4a20-4e4f-8ac5-43c47e85e90e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:15.914810 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652c6562-4a20-4e4f-8ac5-43c47e85e90e admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:15.915276 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652c6562-4a20-4e4f-8ac5-43c47e85e90e admin admin] PATH_INFO: `/v3/users/f44c8dee7121485b81d22764abe990bc` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:15.916906 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652c6562-4a20-4e4f-8ac5-43c47e85e90e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f44c8dee7121485b81d22764abe990bc)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:15.926116 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652c6562-4a20-4e4f-8ac5-43c47e85e90e admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:16.276259 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-652c6562-4a20-4e4f-8ac5-43c47e85e90e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f44c8dee7121485b81d22764abe990bc'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:16.321522 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-652c6562-4a20-4e4f-8ac5-43c47e85e90e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f44c8dee7121485b81d22764abe990bc'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:16.325863 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-652c6562-4a20-4e4f-8ac5-43c47e85e90e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f44c8dee7121485b81d22764abe990bc'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:16.355173 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-652c6562-4a20-4e4f-8ac5-43c47e85e90e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f44c8dee7121485b81d22764abe990bc'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:16.356607 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-652c6562-4a20-4e4f-8ac5-43c47e85e90e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f44c8dee7121485b81d22764abe990bc'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:16.487557 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 810/1649] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:25:15 2022] DELETE /identity/v3/users/f44c8dee7121485b81d22764abe990bc => generated 0 bytes in 769 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:25:16.494407 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9cab0d-2fad-437a-8012-cd7b60c66dab None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:16.494904 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9cab0d-2fad-437a-8012-cd7b60c66dab None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:16.495130 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9cab0d-2fad-437a-8012-cd7b60c66dab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:16.518602 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-fd9cab0d-2fad-437a-8012-cd7b60c66dab None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:25:16.519959 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-fd9cab0d-2fad-437a-8012-cd7b60c66dab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:16.646996 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:16.667421 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 840/1650] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:25:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:25:16.728632 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652c6562-4a20-4e4f-8ac5-43c47e85e90e admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:16.732403 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-652c6562-4a20-4e4f-8ac5-43c47e85e90e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:16.767752 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57af4c91-013b-41a4-9294-458c6a368689 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:16.769154 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57af4c91-013b-41a4-9294-458c6a368689 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:16.769154 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57af4c91-013b-41a4-9294-458c6a368689 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:16.770874 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 841/1651] 10.222.0.25 () {54 vars in 912 bytes} [Thu Aug 4 21:25:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 21:25:16.786843 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bb55e9-df04-46c6-9612-608b28c4a10c None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:16.787117 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bb55e9-df04-46c6-9612-608b28c4a10c None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:16.787553 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bb55e9-df04-46c6-9612-608b28c4a10c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:16.815600 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:16.897455 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652c6562-4a20-4e4f-8ac5-43c47e85e90e admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:16.900561 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-18bb55e9-df04-46c6-9612-608b28c4a10c None None] MFA Rules not processed for user `08fcb91ee54344b5bb3870c585e25fdc`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:25:16.906921 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-18bb55e9-df04-46c6-9612-608b28c4a10c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:16.915805 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aad5912-a7aa-4df0-8468-4c741137f324 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:16.916872 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad5912-a7aa-4df0-8468-4c741137f324 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:16.917366 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad5912-a7aa-4df0-8468-4c741137f324 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:16.917836 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad5912-a7aa-4df0-8468-4c741137f324 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:16.933852 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 842/1652] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Aug 4 21:25:16 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:25:16.958127 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aad5912-a7aa-4df0-8468-4c741137f324 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:16.960912 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aad5912-a7aa-4df0-8468-4c741137f324 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:16.976995 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 811/1653] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:25:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:25:17.006761 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bb55e9-df04-46c6-9612-608b28c4a10c None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:17.021300 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bb55e9-df04-46c6-9612-608b28c4a10c None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:17.030108 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08872d13-f1df-4243-a971-9112a1052161 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:17.030883 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08872d13-f1df-4243-a971-9112a1052161 service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:17.031868 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08872d13-f1df-4243-a971-9112a1052161 service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:17.031868 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08872d13-f1df-4243-a971-9112a1052161 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:17.045176 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08872d13-f1df-4243-a971-9112a1052161 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:17.047506 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08872d13-f1df-4243-a971-9112a1052161 service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:17.058326 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 843/1654] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:25:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:25:17.228353 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52363e3-5df2-4423-a8bc-ea6c3e4aa24e None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:17.228609 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52363e3-5df2-4423-a8bc-ea6c3e4aa24e None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:17.228912 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52363e3-5df2-4423-a8bc-ea6c3e4aa24e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:17.297054 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-c52363e3-5df2-4423-a8bc-ea6c3e4aa24e None None] MFA Rules not processed for user `4087b29e666c4afaa36118e82eb169d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:25:17.299781 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-c52363e3-5df2-4423-a8bc-ea6c3e4aa24e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:17.481558 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103f13c3-e918-423e-86db-aa8cb934155e None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:17.482126 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103f13c3-e918-423e-86db-aa8cb934155e None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:17.482300 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103f13c3-e918-423e-86db-aa8cb934155e None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:17.483867 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 844/1655] 10.222.0.25 () {54 vars in 921 bytes} [Thu Aug 4 21:25:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 21:25:17.531943 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 812/1656] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:25:17 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:25:17.545488 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8dbd9a-5316-476f-a6ae-e2d3a5118e91 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:17.546392 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8dbd9a-5316-476f-a6ae-e2d3a5118e91 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:17.546836 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8dbd9a-5316-476f-a6ae-e2d3a5118e91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:17.552672 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52363e3-5df2-4423-a8bc-ea6c3e4aa24e None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:17.554934 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-c52363e3-5df2-4423-a8bc-ea6c3e4aa24e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:17.628627 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-8c8dbd9a-5316-476f-a6ae-e2d3a5118e91 None None] MFA Rules not processed for user `67d44a0098ab4c36b14b36e9e59ae75c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:25:17.629890 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-8c8dbd9a-5316-476f-a6ae-e2d3a5118e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:17.866615 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52363e3-5df2-4423-a8bc-ea6c3e4aa24e None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:17.871511 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 845/1657] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Aug 4 21:25:17 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:25:17.884769 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c71b86-02ce-4cdc-b329-26c680678117 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a2ce1df87fa47e5be301916bf3245fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:17.885179 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c71b86-02ce-4cdc-b329-26c680678117 service nova] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:17.885634 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c71b86-02ce-4cdc-b329-26c680678117 service nova] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:17.886067 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c71b86-02ce-4cdc-b329-26c680678117 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:17.901395 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c71b86-02ce-4cdc-b329-26c680678117 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:17.902735 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c71b86-02ce-4cdc-b329-26c680678117 service nova] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:17.925450 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 813/1658] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:25:17 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:25:18.062708 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be4b0f8-f3bc-4ecb-8598-f5ffd9ec5509 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:18.062708 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be4b0f8-f3bc-4ecb-8598-f5ffd9ec5509 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:18.062708 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be4b0f8-f3bc-4ecb-8598-f5ffd9ec5509 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:18.066486 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 846/1659] 10.222.0.25 () {54 vars in 961 bytes} [Thu Aug 4 21:25:18 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:25:18.072615 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0661575-ec0a-45c1-b53b-6925fa216d8b None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:18.072926 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0661575-ec0a-45c1-b53b-6925fa216d8b None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:18.073173 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0661575-ec0a-45c1-b53b-6925fa216d8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:18.138692 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-e0661575-ec0a-45c1-b53b-6925fa216d8b None None] MFA Rules not processed for user `67d44a0098ab4c36b14b36e9e59ae75c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:25:18.145852 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-e0661575-ec0a-45c1-b53b-6925fa216d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:18.191208 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be4b0f8-f3bc-4ecb-8598-f5ffd9ec5509 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:18.192082 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 814/1660] 10.222.0.25 () {58 vars in 1058 bytes} [Thu Aug 4 21:25:18 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:25:18.221722 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be4b0f8-f3bc-4ecb-8598-f5ffd9ec5509 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:18.230402 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5ba7de-70e5-4187-bdd8-dba29bd360db service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67d44a0098ab4c36b14b36e9e59ae75c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:18.231122 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5ba7de-70e5-4187-bdd8-dba29bd360db service placement] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:18.231432 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5ba7de-70e5-4187-bdd8-dba29bd360db service placement] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:18.231700 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5ba7de-70e5-4187-bdd8-dba29bd360db service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:18.260750 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5ba7de-70e5-4187-bdd8-dba29bd360db service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:18.262371 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5ba7de-70e5-4187-bdd8-dba29bd360db service placement] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:18.315656 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 847/1661] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:25:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:25:18.652110 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c47408-e2d6-4f10-ac44-5888f8dbd78e None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:18.652110 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c47408-e2d6-4f10-ac44-5888f8dbd78e None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:18.652110 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c47408-e2d6-4f10-ac44-5888f8dbd78e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:18.764081 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-e5c47408-e2d6-4f10-ac44-5888f8dbd78e None None] MFA Rules not processed for user `ea2f8480bc7d44f6a51e4a1d875a1c7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:25:18.772460 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-e5c47408-e2d6-4f10-ac44-5888f8dbd78e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:18.816947 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5ba7de-70e5-4187-bdd8-dba29bd360db service placement] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:18.841575 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:18.841680 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5ba7de-70e5-4187-bdd8-dba29bd360db service placement] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:18.853779 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e418a1-0cdf-44ae-b5a7-90d18ad7592c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67d44a0098ab4c36b14b36e9e59ae75c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:18.855948 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e418a1-0cdf-44ae-b5a7-90d18ad7592c service placement] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:18.856334 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e418a1-0cdf-44ae-b5a7-90d18ad7592c service placement] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:18.856689 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e418a1-0cdf-44ae-b5a7-90d18ad7592c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:18.865759 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-51e418a1-0cdf-44ae-b5a7-90d18ad7592c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:18.943715 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e418a1-0cdf-44ae-b5a7-90d18ad7592c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:18.943715 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e418a1-0cdf-44ae-b5a7-90d18ad7592c service placement] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:18.978998 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 815/1662] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:25:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Aug 04 21:25:18.988065 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d26750-3d18-474d-a7b4-7607b69493b4 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:18.988583 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d26750-3d18-474d-a7b4-7607b69493b4 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:18.988933 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d26750-3d18-474d-a7b4-7607b69493b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:19.003752 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 848/1663] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:25:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:25:19.004934 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e418a1-0cdf-44ae-b5a7-90d18ad7592c service placement] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:19.016969 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-51e418a1-0cdf-44ae-b5a7-90d18ad7592c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:19.044657 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e418a1-0cdf-44ae-b5a7-90d18ad7592c service placement] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:19.046456 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-b2d26750-3d18-474d-a7b4-7607b69493b4 None None] MFA Rules not processed for user `3d5f53509b2f42269cd26e77c79ad8be`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:25:19.051231 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-b2d26750-3d18-474d-a7b4-7607b69493b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:19.052730 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fe981a-2897-4ea2-920a-d76af61aeb32 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a2ce1df87fa47e5be301916bf3245fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:19.053651 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fe981a-2897-4ea2-920a-d76af61aeb32 service nova] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:19.054138 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fe981a-2897-4ea2-920a-d76af61aeb32 service nova] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:19.055206 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fe981a-2897-4ea2-920a-d76af61aeb32 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:19.066361 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fe981a-2897-4ea2-920a-d76af61aeb32 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:19.069872 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fe981a-2897-4ea2-920a-d76af61aeb32 service nova] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:19.119102 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 849/1664] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:25:19 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:25:19.123054 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fe981a-2897-4ea2-920a-d76af61aeb32 service nova] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:19.130254 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-12fe981a-2897-4ea2-920a-d76af61aeb32 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:19.179210 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fe981a-2897-4ea2-920a-d76af61aeb32 service nova] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:19.185912 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ae08e1-f9eb-4fcb-ae1e-3f6525ccaba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:19.186977 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ae08e1-f9eb-4fcb-ae1e-3f6525ccaba1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:19.187840 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ae08e1-f9eb-4fcb-ae1e-3f6525ccaba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:19.188280 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ae08e1-f9eb-4fcb-ae1e-3f6525ccaba1 admin admin] PATH_INFO: `/v3/projects/9c97e87ed5cc4cd5b928e4b8e5be8053` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:19.193537 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ae08e1-f9eb-4fcb-ae1e-3f6525ccaba1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c97e87ed5cc4cd5b928e4b8e5be8053)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:19.195958 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ae08e1-f9eb-4fcb-ae1e-3f6525ccaba1 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:19.290928 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 816/1665] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:25:18 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:25:19.421765 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-a8ae08e1-f9eb-4fcb-ae1e-3f6525ccaba1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c97e87ed5cc4cd5b928e4b8e5be8053', 'request_id': 'req-a8ae08e1-f9eb-4fcb-ae1e-3f6525ccaba1'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:19.450112 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-a8ae08e1-f9eb-4fcb-ae1e-3f6525ccaba1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c97e87ed5cc4cd5b928e4b8e5be8053', 'request_id': 'req-a8ae08e1-f9eb-4fcb-ae1e-3f6525ccaba1'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:19.650479 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-a8ae08e1-f9eb-4fcb-ae1e-3f6525ccaba1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c97e87ed5cc4cd5b928e4b8e5be8053', 'request_id': 'req-a8ae08e1-f9eb-4fcb-ae1e-3f6525ccaba1'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:19.676145 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-a8ae08e1-f9eb-4fcb-ae1e-3f6525ccaba1 admin admin] The token cache is being invalidate because project 9c97e87ed5cc4cd5b928e4b8e5be8053 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:25:19.676506 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-a8ae08e1-f9eb-4fcb-ae1e-3f6525ccaba1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:19.679640 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-a8ae08e1-f9eb-4fcb-ae1e-3f6525ccaba1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:19.688830 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 850/1666] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:25:19 2022] DELETE /identity/v3/projects/9c97e87ed5cc4cd5b928e4b8e5be8053 => generated 0 bytes in 565 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:25:19.691142 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ad2d84-ae9e-409d-bf70-4c851242e231 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:19.692123 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ad2d84-ae9e-409d-bf70-4c851242e231 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:19.692629 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ad2d84-ae9e-409d-bf70-4c851242e231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:19.717150 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-63ad2d84-ae9e-409d-bf70-4c851242e231 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:25:19.721144 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-63ad2d84-ae9e-409d-bf70-4c851242e231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:19.847816 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 817/1667] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:25:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:25:19.849530 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd69035d-14fb-4f84-8f6f-ba06d3b8c179 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:19.850034 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd69035d-14fb-4f84-8f6f-ba06d3b8c179 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:19.850113 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd69035d-14fb-4f84-8f6f-ba06d3b8c179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:19.869952 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:19.902397 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-fd69035d-14fb-4f84-8f6f-ba06d3b8c179 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:25:19.904520 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-fd69035d-14fb-4f84-8f6f-ba06d3b8c179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:19.932329 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 851/1668] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:25:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:25:19.939169 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ad2d84-ae9e-409d-bf70-4c851242e231 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:19.958909 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ad2d84-ae9e-409d-bf70-4c851242e231 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:19.965838 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bd16c8-a08a-4bc2-9cbe-84a1caf015bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:19.966931 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd16c8-a08a-4bc2-9cbe-84a1caf015bf admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:19.967482 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd16c8-a08a-4bc2-9cbe-84a1caf015bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:19.972305 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd16c8-a08a-4bc2-9cbe-84a1caf015bf admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:19.972305 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bd16c8-a08a-4bc2-9cbe-84a1caf015bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:25:19.972834 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bd16c8-a08a-4bc2-9cbe-84a1caf015bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:19.974902 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bd16c8-a08a-4bc2-9cbe-84a1caf015bf admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:19.994525 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 818/1669] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Aug 4 21:25:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:25:19.996313 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd69035d-14fb-4f84-8f6f-ba06d3b8c179 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:20.036523 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd69035d-14fb-4f84-8f6f-ba06d3b8c179 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:20.060902 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59186b5d-7d14-4825-a77f-37bb5d2f4577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:20.061871 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59186b5d-7d14-4825-a77f-37bb5d2f4577 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:20.062575 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59186b5d-7d14-4825-a77f-37bb5d2f4577 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:20.063000 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59186b5d-7d14-4825-a77f-37bb5d2f4577 admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:20.063902 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59186b5d-7d14-4825-a77f-37bb5d2f4577 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:20.066077 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59186b5d-7d14-4825-a77f-37bb5d2f4577 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:20.222879 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 852/1670] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:25:19 2022] POST /identity/v3/projects => generated 373 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:25:20.227954 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bd16c8-a08a-4bc2-9cbe-84a1caf015bf admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:20.247961 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bd16c8-a08a-4bc2-9cbe-84a1caf015bf admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:20.337227 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:20.346916 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c269e7d4-85bf-4c2a-a2d6-f27df0f9e934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:20.347583 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c269e7d4-85bf-4c2a-a2d6-f27df0f9e934 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:20.348106 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c269e7d4-85bf-4c2a-a2d6-f27df0f9e934 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:20.348462 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c269e7d4-85bf-4c2a-a2d6-f27df0f9e934 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:20.349772 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c269e7d4-85bf-4c2a-a2d6-f27df0f9e934 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:20.351946 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c269e7d4-85bf-4c2a-a2d6-f27df0f9e934 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:20.528656 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 819/1671] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:25:20 2022] POST /identity/v3/users => generated 287 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:25:20.529439 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59186b5d-7d14-4825-a77f-37bb5d2f4577 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:20.554578 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59186b5d-7d14-4825-a77f-37bb5d2f4577 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:20.586807 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f16affd-88fe-4a09-805c-81f8bd6e351d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:20.591660 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f16affd-88fe-4a09-805c-81f8bd6e351d admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:20.619252 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f16affd-88fe-4a09-805c-81f8bd6e351d admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:20.619252 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f16affd-88fe-4a09-805c-81f8bd6e351d admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:20.619252 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f16affd-88fe-4a09-805c-81f8bd6e351d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:20.619252 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f16affd-88fe-4a09-805c-81f8bd6e351d admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:20.622495 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 853/1672] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:25:20 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:25:20.637876 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c269e7d4-85bf-4c2a-a2d6-f27df0f9e934 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:20.652133 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c269e7d4-85bf-4c2a-a2d6-f27df0f9e934 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:20.661458 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8234f594-e7ce-435d-b712-b957c976b079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:20.662041 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8234f594-e7ce-435d-b712-b957c976b079 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:20.662295 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8234f594-e7ce-435d-b712-b957c976b079 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:20.662567 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8234f594-e7ce-435d-b712-b957c976b079 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:20.663269 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8234f594-e7ce-435d-b712-b957c976b079 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:20.664926 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8234f594-e7ce-435d-b712-b957c976b079 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:20.697911 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 820/1673] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:25:20 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:25:20.702164 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f16affd-88fe-4a09-805c-81f8bd6e351d admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:20.704525 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-0f16affd-88fe-4a09-805c-81f8bd6e351d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:20.708833 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8234f594-e7ce-435d-b712-b957c976b079 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:20.728176 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f16affd-88fe-4a09-805c-81f8bd6e351d admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:20.758123 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f1f0e4-20f9-4b0b-b29b-2e4d90d20181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:20.759067 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f1f0e4-20f9-4b0b-b29b-2e4d90d20181 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:20.759399 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f1f0e4-20f9-4b0b-b29b-2e4d90d20181 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:20.759740 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f1f0e4-20f9-4b0b-b29b-2e4d90d20181 admin admin] PATH_INFO: `/v3/users/80edbdb9f17d478e943729bdde2ca036` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:20.762431 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f1f0e4-20f9-4b0b-b29b-2e4d90d20181 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80edbdb9f17d478e943729bdde2ca036)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:20.763513 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: WARNING py.warnings [None req-04f1f0e4-20f9-4b0b-b29b-2e4d90d20181 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 21:25:20.763513 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: warnings.warn(msg) Aug 04 21:25:20.763513 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: Aug 04 21:25:20.768516 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f1f0e4-20f9-4b0b-b29b-2e4d90d20181 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:20.770116 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8234f594-e7ce-435d-b712-b957c976b079 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:20.790019 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd63f8a-8e32-46a4-8c4d-4fb47b9f4d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:20.791114 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd63f8a-8e32-46a4-8c4d-4fb47b9f4d1f admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:20.792794 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd63f8a-8e32-46a4-8c4d-4fb47b9f4d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:20.793801 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd63f8a-8e32-46a4-8c4d-4fb47b9f4d1f admin admin] PATH_INFO: `/v3/projects/fc7041e739f54122a0331445e91e6dba/users/ecbd9ed2a4c64c1e9998189af9e574d0/roles/8feebc60363c4feb870718b1721e5324` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:20.846766 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd63f8a-8e32-46a4-8c4d-4fb47b9f4d1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc7041e739f54122a0331445e91e6dba, user_id=ecbd9ed2a4c64c1e9998189af9e574d0, role_id=8feebc60363c4feb870718b1721e5324)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:20.850325 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd63f8a-8e32-46a4-8c4d-4fb47b9f4d1f admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:20.977338 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 821/1674] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:25:20 2022] PUT /identity/v3/projects/fc7041e739f54122a0331445e91e6dba/users/ecbd9ed2a4c64c1e9998189af9e574d0/roles/8feebc60363c4feb870718b1721e5324 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:25:20.979230 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589904e9-977a-438a-ab97-68fbbecc64c0 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:20.980990 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589904e9-977a-438a-ab97-68fbbecc64c0 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:20.981510 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589904e9-977a-438a-ab97-68fbbecc64c0 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:20.999581 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 822/1675] 10.222.0.25 () {54 vars in 912 bytes} [Thu Aug 4 21:25:20 2022] GET /identity => generated 270 bytes in 20 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:25:21.006087 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ea80db-9d9c-468c-8b70-0987471b30a0 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:21.006543 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ea80db-9d9c-468c-8b70-0987471b30a0 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:21.010321 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ea80db-9d9c-468c-8b70-0987471b30a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:21.086491 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-04f1f0e4-20f9-4b0b-b29b-2e4d90d20181 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80edbdb9f17d478e943729bdde2ca036'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:21.116869 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-b7ea80db-9d9c-468c-8b70-0987471b30a0 None None] MFA Rules not processed for user `ecbd9ed2a4c64c1e9998189af9e574d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:25:21.118933 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-b7ea80db-9d9c-468c-8b70-0987471b30a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:21.151144 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-04f1f0e4-20f9-4b0b-b29b-2e4d90d20181 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80edbdb9f17d478e943729bdde2ca036'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:21.156587 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-04f1f0e4-20f9-4b0b-b29b-2e4d90d20181 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80edbdb9f17d478e943729bdde2ca036'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:21.159334 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-04f1f0e4-20f9-4b0b-b29b-2e4d90d20181 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80edbdb9f17d478e943729bdde2ca036'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:21.241652 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 823/1676] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:25:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:25:21.245424 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad63336-f1c3-42d8-99c7-d28a4e33712b None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:21.245734 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad63336-f1c3-42d8-99c7-d28a4e33712b None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:21.246016 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad63336-f1c3-42d8-99c7-d28a4e33712b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:21.278897 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-0ad63336-f1c3-42d8-99c7-d28a4e33712b None None] MFA Rules not processed for user `08fcb91ee54344b5bb3870c585e25fdc`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:25:21.279865 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-04f1f0e4-20f9-4b0b-b29b-2e4d90d20181 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80edbdb9f17d478e943729bdde2ca036'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:21.281162 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-0ad63336-f1c3-42d8-99c7-d28a4e33712b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:21.328657 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 854/1677] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:25:20 2022] DELETE /identity/v3/users/80edbdb9f17d478e943729bdde2ca036 => generated 0 bytes in 627 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:25:21.330170 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f1f0e4-20f9-4b0b-b29b-2e4d90d20181 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:21.333943 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-04f1f0e4-20f9-4b0b-b29b-2e4d90d20181 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:21.450886 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 824/1678] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Aug 4 21:25:21 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:25:21.522397 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad63336-f1c3-42d8-99c7-d28a4e33712b None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:21.522397 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-0ad63336-f1c3-42d8-99c7-d28a4e33712b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:21.535586 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad63336-f1c3-42d8-99c7-d28a4e33712b None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:21.566316 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f1f0e4-20f9-4b0b-b29b-2e4d90d20181 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:21.586842 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13348950-ce08-435d-883d-a237815057d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a2ce1df87fa47e5be301916bf3245fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:21.586842 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13348950-ce08-435d-883d-a237815057d9 service nova] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:21.586842 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13348950-ce08-435d-883d-a237815057d9 service nova] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:21.586842 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13348950-ce08-435d-883d-a237815057d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:21.651149 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:21.686602 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13348950-ce08-435d-883d-a237815057d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:21.691509 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13348950-ce08-435d-883d-a237815057d9 service nova] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:21.700152 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:21.705232 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af98293-4d24-4821-815c-9f1001a7a18e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:21.706491 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af98293-4d24-4821-815c-9f1001a7a18e service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:21.707209 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af98293-4d24-4821-815c-9f1001a7a18e service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:21.707760 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af98293-4d24-4821-815c-9f1001a7a18e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:21.720905 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 855/1679] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:25:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:25:21.740845 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af98293-4d24-4821-815c-9f1001a7a18e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:21.745681 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af98293-4d24-4821-815c-9f1001a7a18e service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:21.774372 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 825/1680] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:25:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:25:23.310582 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13348950-ce08-435d-883d-a237815057d9 service nova] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:23.315025 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-13348950-ce08-435d-883d-a237815057d9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:23.679805 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13348950-ce08-435d-883d-a237815057d9 service nova] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:23.690660 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dcce8e-c2fe-45cd-a903-b572cc31ee43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:23.691665 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dcce8e-c2fe-45cd-a903-b572cc31ee43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:23.691953 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dcce8e-c2fe-45cd-a903-b572cc31ee43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:23.692235 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dcce8e-c2fe-45cd-a903-b572cc31ee43 admin admin] PATH_INFO: `/v3/projects/a1672ce405d74c32919c98f4ae3d4c65` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:23.694058 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dcce8e-c2fe-45cd-a903-b572cc31ee43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1672ce405d74c32919c98f4ae3d4c65)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:23.696665 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dcce8e-c2fe-45cd-a903-b572cc31ee43 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:23.930164 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-32dcce8e-c2fe-45cd-a903-b572cc31ee43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1672ce405d74c32919c98f4ae3d4c65', 'request_id': 'req-32dcce8e-c2fe-45cd-a903-b572cc31ee43'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:23.961783 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-32dcce8e-c2fe-45cd-a903-b572cc31ee43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1672ce405d74c32919c98f4ae3d4c65', 'request_id': 'req-32dcce8e-c2fe-45cd-a903-b572cc31ee43'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:24.026183 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06f2c86-943c-4d81-8954-7b47283771c0 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:24.031904 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06f2c86-943c-4d81-8954-7b47283771c0 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:24.031904 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06f2c86-943c-4d81-8954-7b47283771c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:24.120065 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-a06f2c86-943c-4d81-8954-7b47283771c0 None None] MFA Rules not processed for user `9b29c3f2c29d4e2aba552c144cff94f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:25:24.123708 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-32dcce8e-c2fe-45cd-a903-b572cc31ee43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1672ce405d74c32919c98f4ae3d4c65', 'request_id': 'req-32dcce8e-c2fe-45cd-a903-b572cc31ee43'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:24.123923 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-a06f2c86-943c-4d81-8954-7b47283771c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:24.152809 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-32dcce8e-c2fe-45cd-a903-b572cc31ee43 admin admin] The token cache is being invalidate because project a1672ce405d74c32919c98f4ae3d4c65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:25:24.161996 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-32dcce8e-c2fe-45cd-a903-b572cc31ee43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:24.161996 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-32dcce8e-c2fe-45cd-a903-b572cc31ee43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:24.162701 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 856/1681] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:25:23 2022] DELETE /identity/v3/projects/a1672ce405d74c32919c98f4ae3d4c65 => generated 0 bytes in 852 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:25:24.186236 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dcce8e-c2fe-45cd-a903-b572cc31ee43 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:24.202965 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-32dcce8e-c2fe-45cd-a903-b572cc31ee43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:24.305967 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 826/1682] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:25:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:25:24.344353 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06f2c86-943c-4d81-8954-7b47283771c0 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:24.346679 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-a06f2c86-943c-4d81-8954-7b47283771c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:24.394414 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dcce8e-c2fe-45cd-a903-b572cc31ee43 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:24.424378 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047b7fea-9a04-488e-bed6-611980dcff58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:24.425220 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b7fea-9a04-488e-bed6-611980dcff58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:24.425624 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b7fea-9a04-488e-bed6-611980dcff58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:24.426005 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b7fea-9a04-488e-bed6-611980dcff58 admin admin] PATH_INFO: `/v3/users/4488f60b29364a02a1c2423e1c8c445e` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:24.432474 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047b7fea-9a04-488e-bed6-611980dcff58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4488f60b29364a02a1c2423e1c8c445e)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:24.435026 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047b7fea-9a04-488e-bed6-611980dcff58 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:24.708699 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06f2c86-943c-4d81-8954-7b47283771c0 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:24.720145 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66459eb-5e06-4767-92c6-48b6cff61390 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a2ce1df87fa47e5be301916bf3245fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:24.721377 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66459eb-5e06-4767-92c6-48b6cff61390 service nova] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:24.721809 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66459eb-5e06-4767-92c6-48b6cff61390 service nova] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:24.722227 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66459eb-5e06-4767-92c6-48b6cff61390 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:24.733377 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-047b7fea-9a04-488e-bed6-611980dcff58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4488f60b29364a02a1c2423e1c8c445e'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:24.739955 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:24.749986 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66459eb-5e06-4767-92c6-48b6cff61390 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:24.755529 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66459eb-5e06-4767-92c6-48b6cff61390 service nova] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:24.773091 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 827/1683] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:25:24 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:25:24.775113 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-047b7fea-9a04-488e-bed6-611980dcff58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4488f60b29364a02a1c2423e1c8c445e'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:24.775938 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-047b7fea-9a04-488e-bed6-611980dcff58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4488f60b29364a02a1c2423e1c8c445e'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:24.776565 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-047b7fea-9a04-488e-bed6-611980dcff58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4488f60b29364a02a1c2423e1c8c445e'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:24.976085 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-047b7fea-9a04-488e-bed6-611980dcff58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4488f60b29364a02a1c2423e1c8c445e'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:24.998954 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 857/1684] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:25:24 2022] DELETE /identity/v3/users/4488f60b29364a02a1c2423e1c8c445e => generated 0 bytes in 821 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:25:25.358481 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959793c7-3774-44ac-ab16-a015a4cb2dd6 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:25.358481 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959793c7-3774-44ac-ab16-a015a4cb2dd6 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:25.358481 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959793c7-3774-44ac-ab16-a015a4cb2dd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:25.477750 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-959793c7-3774-44ac-ab16-a015a4cb2dd6 None None] MFA Rules not processed for user `23ea380385f649628cc27e3603bc700c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:25:25.481602 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-959793c7-3774-44ac-ab16-a015a4cb2dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:25.565751 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:25.677907 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 828/1685] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Aug 4 21:25:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:25:26.132562 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047b7fea-9a04-488e-bed6-611980dcff58 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:26.134122 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-047b7fea-9a04-488e-bed6-611980dcff58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:26.221606 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047b7fea-9a04-488e-bed6-611980dcff58 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:26.234846 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143cff15-5dfb-491f-a6ce-dfee515ee87a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:26.237989 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143cff15-5dfb-491f-a6ce-dfee515ee87a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:26.238344 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143cff15-5dfb-491f-a6ce-dfee515ee87a admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:26.238600 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143cff15-5dfb-491f-a6ce-dfee515ee87a admin admin] PATH_INFO: `/v3/projects/bc03a87905644bf68b2d3fcef84cadac` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:26.242107 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143cff15-5dfb-491f-a6ce-dfee515ee87a admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc03a87905644bf68b2d3fcef84cadac)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:26.245196 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143cff15-5dfb-491f-a6ce-dfee515ee87a admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:26.263805 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959793c7-3774-44ac-ab16-a015a4cb2dd6 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:26.265899 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-959793c7-3774-44ac-ab16-a015a4cb2dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:26.381402 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959793c7-3774-44ac-ab16-a015a4cb2dd6 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:26.408257 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee301045-b3ad-410d-bfc8-c020ce7a854c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58935367c0ad445f8477cdac0df04d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:26.411970 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee301045-b3ad-410d-bfc8-c020ce7a854c service swift] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:26.412290 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee301045-b3ad-410d-bfc8-c020ce7a854c service swift] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:26.412569 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee301045-b3ad-410d-bfc8-c020ce7a854c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:26.489326 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee301045-b3ad-410d-bfc8-c020ce7a854c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:26.489326 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee301045-b3ad-410d-bfc8-c020ce7a854c service swift] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:26.491933 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-143cff15-5dfb-491f-a6ce-dfee515ee87a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc03a87905644bf68b2d3fcef84cadac', 'request_id': 'req-143cff15-5dfb-491f-a6ce-dfee515ee87a'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:26.501784 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-143cff15-5dfb-491f-a6ce-dfee515ee87a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc03a87905644bf68b2d3fcef84cadac', 'request_id': 'req-143cff15-5dfb-491f-a6ce-dfee515ee87a'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:26.518318 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 829/1686] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Aug 4 21:25:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 21:25:26.519002 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee301045-b3ad-410d-bfc8-c020ce7a854c service swift] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:26.528312 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-ee301045-b3ad-410d-bfc8-c020ce7a854c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:26.605912 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee301045-b3ad-410d-bfc8-c020ce7a854c service swift] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:26.633584 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-143cff15-5dfb-491f-a6ce-dfee515ee87a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc03a87905644bf68b2d3fcef84cadac', 'request_id': 'req-143cff15-5dfb-491f-a6ce-dfee515ee87a'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:26.696226 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-143cff15-5dfb-491f-a6ce-dfee515ee87a admin admin] The token cache is being invalidate because project bc03a87905644bf68b2d3fcef84cadac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:25:26.696455 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-143cff15-5dfb-491f-a6ce-dfee515ee87a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:26.704506 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-143cff15-5dfb-491f-a6ce-dfee515ee87a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:26.710282 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 858/1687] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:25:26 2022] DELETE /identity/v3/projects/bc03a87905644bf68b2d3fcef84cadac => generated 0 bytes in 578 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:25:26.721646 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5f2a04-2323-436d-92e8-5f9023ea7b29 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:26.722185 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5f2a04-2323-436d-92e8-5f9023ea7b29 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:26.727846 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5f2a04-2323-436d-92e8-5f9023ea7b29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:26.796827 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-8c5f2a04-2323-436d-92e8-5f9023ea7b29 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:25:26.802198 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-8c5f2a04-2323-436d-92e8-5f9023ea7b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:26.957821 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96821c99-0e8a-4f87-95b4-2cf3d3ea4fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:26.958846 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96821c99-0e8a-4f87-95b4-2cf3d3ea4fc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:26.959413 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96821c99-0e8a-4f87-95b4-2cf3d3ea4fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:26.959904 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96821c99-0e8a-4f87-95b4-2cf3d3ea4fc6 admin admin] PATH_INFO: `/v3/users/ecbd9ed2a4c64c1e9998189af9e574d0` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:26.978284 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96821c99-0e8a-4f87-95b4-2cf3d3ea4fc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecbd9ed2a4c64c1e9998189af9e574d0)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:26.980363 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96821c99-0e8a-4f87-95b4-2cf3d3ea4fc6 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:27.031647 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 859/1688] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:25:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:25:27.035129 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fa744a-fd95-4c7e-bfb8-762fb2c42dd7 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:27.036727 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fa744a-fd95-4c7e-bfb8-762fb2c42dd7 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:27.037251 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fa744a-fd95-4c7e-bfb8-762fb2c42dd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:27.100162 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-13fa744a-fd95-4c7e-bfb8-762fb2c42dd7 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:25:27.101910 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-13fa744a-fd95-4c7e-bfb8-762fb2c42dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:27.122733 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 860/1689] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:25:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:25:27.129397 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fa744a-fd95-4c7e-bfb8-762fb2c42dd7 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:27.157120 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fa744a-fd95-4c7e-bfb8-762fb2c42dd7 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:27.170421 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d51fa6a-3ad1-4136-a580-c2f8e818b050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:27.171307 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d51fa6a-3ad1-4136-a580-c2f8e818b050 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:27.171774 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d51fa6a-3ad1-4136-a580-c2f8e818b050 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:27.172400 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d51fa6a-3ad1-4136-a580-c2f8e818b050 admin admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:27.173881 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d51fa6a-3ad1-4136-a580-c2f8e818b050 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:25:27.174333 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d51fa6a-3ad1-4136-a580-c2f8e818b050 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:27.176162 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d51fa6a-3ad1-4136-a580-c2f8e818b050 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:27.195907 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 861/1690] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Aug 4 21:25:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:25:27.200993 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d51fa6a-3ad1-4136-a580-c2f8e818b050 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:27.230044 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d51fa6a-3ad1-4136-a580-c2f8e818b050 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:27.244523 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911fc7a3-a4a0-4106-846e-606034451267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:27.250912 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911fc7a3-a4a0-4106-846e-606034451267 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:27.250912 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911fc7a3-a4a0-4106-846e-606034451267 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:27.250912 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911fc7a3-a4a0-4106-846e-606034451267 admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:27.250912 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911fc7a3-a4a0-4106-846e-606034451267 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:27.256967 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911fc7a3-a4a0-4106-846e-606034451267 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:27.502830 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-96821c99-0e8a-4f87-95b4-2cf3d3ea4fc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecbd9ed2a4c64c1e9998189af9e574d0'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:27.520442 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:27.560401 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-96821c99-0e8a-4f87-95b4-2cf3d3ea4fc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecbd9ed2a4c64c1e9998189af9e574d0'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:27.561706 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-96821c99-0e8a-4f87-95b4-2cf3d3ea4fc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecbd9ed2a4c64c1e9998189af9e574d0'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:27.582713 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-96821c99-0e8a-4f87-95b4-2cf3d3ea4fc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecbd9ed2a4c64c1e9998189af9e574d0'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:27.588931 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-96821c99-0e8a-4f87-95b4-2cf3d3ea4fc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecbd9ed2a4c64c1e9998189af9e574d0'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:27.589380 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 862/1691] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:25:27 2022] POST /identity/v3/projects => generated 379 bytes in 389 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:25:27.594106 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911fc7a3-a4a0-4106-846e-606034451267 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:27.600456 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-911fc7a3-a4a0-4106-846e-606034451267 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:27.819136 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 830/1692] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:25:26 2022] DELETE /identity/v3/users/ecbd9ed2a4c64c1e9998189af9e574d0 => generated 0 bytes in 1297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:25:27.820959 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c692228a-0b99-4947-bbaa-17336ae1d312 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:27.825655 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c692228a-0b99-4947-bbaa-17336ae1d312 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:27.826172 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c692228a-0b99-4947-bbaa-17336ae1d312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:27.906198 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-c692228a-0b99-4947-bbaa-17336ae1d312 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:25:27.908328 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-c692228a-0b99-4947-bbaa-17336ae1d312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:27.975272 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911fc7a3-a4a0-4106-846e-606034451267 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:28.172879 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 831/1693] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:25:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:25:28.198891 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64b7a1b-402d-4f5e-aec7-db2390d50093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:28.204129 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b7a1b-402d-4f5e-aec7-db2390d50093 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:28.210480 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b7a1b-402d-4f5e-aec7-db2390d50093 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:28.210784 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b7a1b-402d-4f5e-aec7-db2390d50093 admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:28.212480 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64b7a1b-402d-4f5e-aec7-db2390d50093 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:28.218033 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64b7a1b-402d-4f5e-aec7-db2390d50093 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:28.240243 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c692228a-0b99-4947-bbaa-17336ae1d312 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:28.240243 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-c692228a-0b99-4947-bbaa-17336ae1d312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:28.406503 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c692228a-0b99-4947-bbaa-17336ae1d312 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:28.419490 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6111ab7e-c544-472a-a760-f21415b25a19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:28.419704 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6111ab7e-c544-472a-a760-f21415b25a19 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:28.419922 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6111ab7e-c544-472a-a760-f21415b25a19 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:28.420299 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6111ab7e-c544-472a-a760-f21415b25a19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:28.449535 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6111ab7e-c544-472a-a760-f21415b25a19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:28.451145 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6111ab7e-c544-472a-a760-f21415b25a19 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:28.465205 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 863/1694] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:25:27 2022] POST /identity/v3/users => generated 290 bytes in 871 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:25:28.466325 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf70f96-46f8-4356-abf1-db9ae94fd945 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:28.466614 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf70f96-46f8-4356-abf1-db9ae94fd945 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:28.466887 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf70f96-46f8-4356-abf1-db9ae94fd945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:28.476783 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 832/1695] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:25:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:25:28.476783 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6111ab7e-c544-472a-a760-f21415b25a19 service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:28.513139 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-dbf70f96-46f8-4356-abf1-db9ae94fd945 None None] MFA Rules not processed for user `08fcb91ee54344b5bb3870c585e25fdc`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:25:28.513296 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6111ab7e-c544-472a-a760-f21415b25a19 service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:28.517948 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-dbf70f96-46f8-4356-abf1-db9ae94fd945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:28.541508 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 864/1696] 10.222.0.90 () {58 vars in 1015 bytes} [Thu Aug 4 21:25:28 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:25:28.543121 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c66676-692e-4a3d-be24-c326b4ca515e None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:28.543614 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c66676-692e-4a3d-be24-c326b4ca515e None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:28.547677 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c66676-692e-4a3d-be24-c326b4ca515e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:28.547815 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e0e3ca-292d-44c1-9dce-645921ee2744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:28.547815 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e0e3ca-292d-44c1-9dce-645921ee2744 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:28.547815 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e0e3ca-292d-44c1-9dce-645921ee2744 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:28.547815 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e0e3ca-292d-44c1-9dce-645921ee2744 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:28.548682 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e0e3ca-292d-44c1-9dce-645921ee2744 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:28.549044 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e0e3ca-292d-44c1-9dce-645921ee2744 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:28.620397 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-62c66676-692e-4a3d-be24-c326b4ca515e None None] MFA Rules not processed for user `966b1056487d435ca83b6a9b15b0ecea`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:25:28.624037 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-62c66676-692e-4a3d-be24-c326b4ca515e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:28.639640 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 833/1697] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:25:28 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:25:28.641175 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e0e3ca-292d-44c1-9dce-645921ee2744 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:28.668291 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e0e3ca-292d-44c1-9dce-645921ee2744 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:28.681905 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50f8af9-d169-422a-925d-9b7f63aa0e8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:28.682785 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50f8af9-d169-422a-925d-9b7f63aa0e8c service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:28.683173 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50f8af9-d169-422a-925d-9b7f63aa0e8c service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:28.683601 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50f8af9-d169-422a-925d-9b7f63aa0e8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:28.699159 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50f8af9-d169-422a-925d-9b7f63aa0e8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:28.702025 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50f8af9-d169-422a-925d-9b7f63aa0e8c service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:28.723056 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 834/1698] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:25:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:25:28.724139 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50f8af9-d169-422a-925d-9b7f63aa0e8c service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:28.754737 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50f8af9-d169-422a-925d-9b7f63aa0e8c service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:28.775480 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5331fb-dc4c-4a92-8127-9bb1c0644cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:28.776199 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5331fb-dc4c-4a92-8127-9bb1c0644cc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:28.776486 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5331fb-dc4c-4a92-8127-9bb1c0644cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:28.776759 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5331fb-dc4c-4a92-8127-9bb1c0644cc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:28.777461 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5331fb-dc4c-4a92-8127-9bb1c0644cc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:28.784562 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5331fb-dc4c-4a92-8127-9bb1c0644cc3 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:28.808936 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 835/1699] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:25:28 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:25:28.812337 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5331fb-dc4c-4a92-8127-9bb1c0644cc3 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:28.835884 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 865/1700] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:25:28 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:25:28.849068 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b418e259-42a9-44f2-87d7-9fa0cafa3082 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:28.849736 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5331fb-dc4c-4a92-8127-9bb1c0644cc3 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:28.850151 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b418e259-42a9-44f2-87d7-9fa0cafa3082 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:28.850899 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b418e259-42a9-44f2-87d7-9fa0cafa3082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:28.922996 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ec72d8-304c-4123-b3ab-ebe7f5d667e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:28.922996 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ec72d8-304c-4123-b3ab-ebe7f5d667e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:28.922996 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ec72d8-304c-4123-b3ab-ebe7f5d667e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:28.922996 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ec72d8-304c-4123-b3ab-ebe7f5d667e8 admin admin] PATH_INFO: `/v3/projects/1d922c64da53449089653c1b53e09216/users/ddeb36626a5f4a0aa4a6ae73d5a30642/roles/8feebc60363c4feb870718b1721e5324` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:28.923681 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ec72d8-304c-4123-b3ab-ebe7f5d667e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d922c64da53449089653c1b53e09216, user_id=ddeb36626a5f4a0aa4a6ae73d5a30642, role_id=8feebc60363c4feb870718b1721e5324)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:28.923761 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-b418e259-42a9-44f2-87d7-9fa0cafa3082 None None] MFA Rules not processed for user `966b1056487d435ca83b6a9b15b0ecea`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:25:28.923761 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-b418e259-42a9-44f2-87d7-9fa0cafa3082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:28.923934 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ec72d8-304c-4123-b3ab-ebe7f5d667e8 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:28.929883 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 866/1701] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:25:28 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:25:28.955068 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b418e259-42a9-44f2-87d7-9fa0cafa3082 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:28.955068 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-b418e259-42a9-44f2-87d7-9fa0cafa3082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:29.063060 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 836/1702] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:25:28 2022] PUT /identity/v3/projects/1d922c64da53449089653c1b53e09216/users/ddeb36626a5f4a0aa4a6ae73d5a30642/roles/8feebc60363c4feb870718b1721e5324 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:25:29.078041 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e998c6-829f-4568-9c08-ef7f3527c616 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:29.082082 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e998c6-829f-4568-9c08-ef7f3527c616 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:29.082082 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e998c6-829f-4568-9c08-ef7f3527c616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:29.163824 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-f9e998c6-829f-4568-9c08-ef7f3527c616 None None] MFA Rules not processed for user `ddeb36626a5f4a0aa4a6ae73d5a30642`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:25:29.165133 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-f9e998c6-829f-4568-9c08-ef7f3527c616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:29.197164 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b418e259-42a9-44f2-87d7-9fa0cafa3082 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:29.398410 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 837/1703] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:25:29 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:25:29.430771 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e998c6-829f-4568-9c08-ef7f3527c616 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:29.437934 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fb5a5d-3b09-478c-9344-44b931396a0f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a2ce1df87fa47e5be301916bf3245fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:29.439340 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fb5a5d-3b09-478c-9344-44b931396a0f service nova] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:29.439883 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fb5a5d-3b09-478c-9344-44b931396a0f service nova] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:29.441172 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fb5a5d-3b09-478c-9344-44b931396a0f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:29.631162 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fb5a5d-3b09-478c-9344-44b931396a0f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:29.633329 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fb5a5d-3b09-478c-9344-44b931396a0f service nova] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:29.694247 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 867/1704] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:25:28 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 750 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:25:29.732829 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e998c6-829f-4568-9c08-ef7f3527c616 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:29.747261 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be4aeab-bfea-4ef8-80b0-05e4112a7dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:29.747979 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be4aeab-bfea-4ef8-80b0-05e4112a7dd0 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:29.748269 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be4aeab-bfea-4ef8-80b0-05e4112a7dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:29.748574 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be4aeab-bfea-4ef8-80b0-05e4112a7dd0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:29.749308 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be4aeab-bfea-4ef8-80b0-05e4112a7dd0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:29.755451 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be4aeab-bfea-4ef8-80b0-05e4112a7dd0 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:29.763470 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:30.011522 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 838/1705] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:25:29 2022] POST /identity/v3/projects => generated 379 bytes in 581 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:25:30.015644 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fb5a5d-3b09-478c-9344-44b931396a0f service nova] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:30.034356 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fb5a5d-3b09-478c-9344-44b931396a0f service nova] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:30.238147 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288db8ea-f7ff-45a1-9496-1899de344ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:30.238818 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288db8ea-f7ff-45a1-9496-1899de344ba1 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:30.239102 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288db8ea-f7ff-45a1-9496-1899de344ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:30.239403 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288db8ea-f7ff-45a1-9496-1899de344ba1 admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:30.242185 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288db8ea-f7ff-45a1-9496-1899de344ba1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:30.249705 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288db8ea-f7ff-45a1-9496-1899de344ba1 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:30.490013 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 868/1706] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:25:30 2022] POST /identity/v3/users => generated 296 bytes in 475 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:25:30.491707 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be4aeab-bfea-4ef8-80b0-05e4112a7dd0 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:30.514428 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be4aeab-bfea-4ef8-80b0-05e4112a7dd0 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:30.531500 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15418082-623e-4372-ab26-90303bcd338f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:30.532894 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15418082-623e-4372-ab26-90303bcd338f admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:30.533334 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15418082-623e-4372-ab26-90303bcd338f admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:30.534102 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15418082-623e-4372-ab26-90303bcd338f admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:30.535101 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15418082-623e-4372-ab26-90303bcd338f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:30.541051 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15418082-623e-4372-ab26-90303bcd338f admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:30.604551 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 839/1707] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:25:30 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:25:30.611904 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288db8ea-f7ff-45a1-9496-1899de344ba1 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:30.661522 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288db8ea-f7ff-45a1-9496-1899de344ba1 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:30.668474 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:30.682244 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be811a4-d8e1-496f-9980-b216ab7292ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:30.683136 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be811a4-d8e1-496f-9980-b216ab7292ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:30.683486 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be811a4-d8e1-496f-9980-b216ab7292ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:30.683802 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be811a4-d8e1-496f-9980-b216ab7292ac admin admin] PATH_INFO: `/v3/projects/83f31a7bb57841b89048f9b89e17dee2/users/ff862c510c3f4014bcd4c4ee41444f1b/roles/a432dd4735944ecfab2fa5ce1705d356` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:30.754312 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be811a4-d8e1-496f-9980-b216ab7292ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=83f31a7bb57841b89048f9b89e17dee2, user_id=ff862c510c3f4014bcd4c4ee41444f1b, role_id=a432dd4735944ecfab2fa5ce1705d356)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:30.758615 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be811a4-d8e1-496f-9980-b216ab7292ac admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:30.926042 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 869/1708] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:25:30 2022] PUT /identity/v3/projects/83f31a7bb57841b89048f9b89e17dee2/users/ff862c510c3f4014bcd4c4ee41444f1b/roles/a432dd4735944ecfab2fa5ce1705d356 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:25:30.930252 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be811a4-d8e1-496f-9980-b216ab7292ac admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:30.931694 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a913e0d-7532-4e10-b901-36f009c23869 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:30.932206 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a913e0d-7532-4e10-b901-36f009c23869 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:30.932542 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a913e0d-7532-4e10-b901-36f009c23869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:30.988514 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-8a913e0d-7532-4e10-b901-36f009c23869 None None] MFA Rules not processed for user `4087b29e666c4afaa36118e82eb169d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:25:30.989792 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-8a913e0d-7532-4e10-b901-36f009c23869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:31.128358 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be811a4-d8e1-496f-9980-b216ab7292ac admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:31.176671 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf426498-450e-4f62-a10a-f920472adbdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:31.182899 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf426498-450e-4f62-a10a-f920472adbdb admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:31.183578 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf426498-450e-4f62-a10a-f920472adbdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:31.183981 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf426498-450e-4f62-a10a-f920472adbdb admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:31.184896 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf426498-450e-4f62-a10a-f920472adbdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:31.188300 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf426498-450e-4f62-a10a-f920472adbdb admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:31.219410 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 870/1709] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:25:30 2022] GET /identity/v3/roles => generated 1420 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:25:31.223659 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf426498-450e-4f62-a10a-f920472adbdb admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:31.247761 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf426498-450e-4f62-a10a-f920472adbdb admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:31.263579 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329f676c-969b-4ab6-809d-46dba018ed17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:31.264412 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329f676c-969b-4ab6-809d-46dba018ed17 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:31.264787 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329f676c-969b-4ab6-809d-46dba018ed17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:31.265141 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329f676c-969b-4ab6-809d-46dba018ed17 admin admin] PATH_INFO: `/v3/projects/83f31a7bb57841b89048f9b89e17dee2/users/ff862c510c3f4014bcd4c4ee41444f1b/roles/a432dd4735944ecfab2fa5ce1705d356` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:31.274881 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329f676c-969b-4ab6-809d-46dba018ed17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83f31a7bb57841b89048f9b89e17dee2, user_id=ff862c510c3f4014bcd4c4ee41444f1b, role_id=a432dd4735944ecfab2fa5ce1705d356)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:31.278222 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329f676c-969b-4ab6-809d-46dba018ed17 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:31.314622 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 871/1710] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:25:31 2022] PUT /identity/v3/projects/83f31a7bb57841b89048f9b89e17dee2/users/ff862c510c3f4014bcd4c4ee41444f1b/roles/a432dd4735944ecfab2fa5ce1705d356 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:25:31.327179 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d404977-cd50-4f3b-a80b-c70710b103e9 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:31.327884 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d404977-cd50-4f3b-a80b-c70710b103e9 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:31.328262 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d404977-cd50-4f3b-a80b-c70710b103e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:31.377653 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 840/1711] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:25:30 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 448 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:25:31.457582 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-6d404977-cd50-4f3b-a80b-c70710b103e9 None None] MFA Rules not processed for user `ff862c510c3f4014bcd4c4ee41444f1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:25:31.459502 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-6d404977-cd50-4f3b-a80b-c70710b103e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:31.740864 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 872/1712] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:25:31 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 401 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:25:31.762528 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a913e0d-7532-4e10-b901-36f009c23869 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:31.767038 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-8a913e0d-7532-4e10-b901-36f009c23869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:31.969453 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a913e0d-7532-4e10-b901-36f009c23869 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:32.031109 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d45f85-4b32-4c9d-9cdc-d5ae24ffdbc1 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:32.031109 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d45f85-4b32-4c9d-9cdc-d5ae24ffdbc1 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:32.031109 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d45f85-4b32-4c9d-9cdc-d5ae24ffdbc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:32.093640 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-93d45f85-4b32-4c9d-9cdc-d5ae24ffdbc1 None None] MFA Rules not processed for user `ea2f8480bc7d44f6a51e4a1d875a1c7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:25:32.097167 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-93d45f85-4b32-4c9d-9cdc-d5ae24ffdbc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:32.120273 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078484e0-94ae-4cff-ae06-65995ad0e7e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e31f2728056b460fa474478eb85ca1d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:32.124329 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078484e0-94ae-4cff-ae06-65995ad0e7e3 service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:32.125379 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078484e0-94ae-4cff-ae06-65995ad0e7e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:32.126970 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078484e0-94ae-4cff-ae06-65995ad0e7e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:32.181199 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078484e0-94ae-4cff-ae06-65995ad0e7e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:32.182827 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078484e0-94ae-4cff-ae06-65995ad0e7e3 service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:32.272027 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 841/1713] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:25:31 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:25:32.293503 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078484e0-94ae-4cff-ae06-65995ad0e7e3 service cinder] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:32.296060 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-078484e0-94ae-4cff-ae06-65995ad0e7e3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:32.467744 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 873/1714] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:25:32 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 455 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:25:32.516507 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078484e0-94ae-4cff-ae06-65995ad0e7e3 service cinder] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:32.524248 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0418c7bf-3667-4908-86c0-36fb2e532209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:32.524666 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0418c7bf-3667-4908-86c0-36fb2e532209 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:32.525053 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0418c7bf-3667-4908-86c0-36fb2e532209 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:32.525688 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0418c7bf-3667-4908-86c0-36fb2e532209 admin admin] PATH_INFO: `/v3/projects/fc7041e739f54122a0331445e91e6dba` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:32.528428 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0418c7bf-3667-4908-86c0-36fb2e532209 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc7041e739f54122a0331445e91e6dba)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:32.530504 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: WARNING py.warnings [None req-0418c7bf-3667-4908-86c0-36fb2e532209 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 21:25:32.530504 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: warnings.warn(msg) Aug 04 21:25:32.530504 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: Aug 04 21:25:32.532582 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0418c7bf-3667-4908-86c0-36fb2e532209 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:32.589812 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d45f85-4b32-4c9d-9cdc-d5ae24ffdbc1 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:32.628273 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d45f85-4b32-4c9d-9cdc-d5ae24ffdbc1 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:32.633710 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539cc6ea-3638-4317-ad0c-84979564f488 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e31f2728056b460fa474478eb85ca1d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:32.635408 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539cc6ea-3638-4317-ad0c-84979564f488 service cinder] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:32.635858 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539cc6ea-3638-4317-ad0c-84979564f488 service cinder] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:32.636298 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539cc6ea-3638-4317-ad0c-84979564f488 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:32.776625 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539cc6ea-3638-4317-ad0c-84979564f488 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:32.778748 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539cc6ea-3638-4317-ad0c-84979564f488 service cinder] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:32.804663 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 874/1715] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:25:32 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:25:32.829553 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-0418c7bf-3667-4908-86c0-36fb2e532209 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc7041e739f54122a0331445e91e6dba', 'request_id': 'req-0418c7bf-3667-4908-86c0-36fb2e532209'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:32.837536 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-0418c7bf-3667-4908-86c0-36fb2e532209 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc7041e739f54122a0331445e91e6dba', 'request_id': 'req-0418c7bf-3667-4908-86c0-36fb2e532209'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:32.970955 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-0418c7bf-3667-4908-86c0-36fb2e532209 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc7041e739f54122a0331445e91e6dba', 'request_id': 'req-0418c7bf-3667-4908-86c0-36fb2e532209'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:33.012952 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-0418c7bf-3667-4908-86c0-36fb2e532209 admin admin] The token cache is being invalidate because project fc7041e739f54122a0331445e91e6dba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:25:33.013405 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-0418c7bf-3667-4908-86c0-36fb2e532209 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:33.014309 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-0418c7bf-3667-4908-86c0-36fb2e532209 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:33.017287 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 842/1716] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:25:32 2022] DELETE /identity/v3/projects/fc7041e739f54122a0331445e91e6dba => generated 0 bytes in 729 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:25:33.022079 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca3483e-2459-4270-a4f2-8efc52c86414 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:33.022736 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca3483e-2459-4270-a4f2-8efc52c86414 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:33.023274 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca3483e-2459-4270-a4f2-8efc52c86414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:33.063824 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-aca3483e-2459-4270-a4f2-8efc52c86414 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:25:33.065292 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-aca3483e-2459-4270-a4f2-8efc52c86414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:33.256194 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 875/1717] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:25:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:25:33.262890 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416566c0-ef36-414a-bce4-8e97a4d80519 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:33.263185 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416566c0-ef36-414a-bce4-8e97a4d80519 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:33.263537 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416566c0-ef36-414a-bce4-8e97a4d80519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:33.297293 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-416566c0-ef36-414a-bce4-8e97a4d80519 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:25:33.300049 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-416566c0-ef36-414a-bce4-8e97a4d80519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:33.348966 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 843/1718] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:25:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:25:33.354886 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca3483e-2459-4270-a4f2-8efc52c86414 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:33.379127 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca3483e-2459-4270-a4f2-8efc52c86414 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:33.406210 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fcba5b-351c-4804-8eaf-4ef0c85fab45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:33.416612 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fcba5b-351c-4804-8eaf-4ef0c85fab45 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:33.416976 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fcba5b-351c-4804-8eaf-4ef0c85fab45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:33.418977 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fcba5b-351c-4804-8eaf-4ef0c85fab45 admin admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:33.427098 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fcba5b-351c-4804-8eaf-4ef0c85fab45 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:25:33.427538 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fcba5b-351c-4804-8eaf-4ef0c85fab45 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:33.440200 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fcba5b-351c-4804-8eaf-4ef0c85fab45 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:33.519406 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416566c0-ef36-414a-bce4-8e97a4d80519 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:33.545482 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 876/1719] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Aug 4 21:25:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:25:33.567487 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416566c0-ef36-414a-bce4-8e97a4d80519 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:33.620294 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecbeac4-a2c5-433b-a91c-cca3c8e963c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:33.625216 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c597cca-da38-48cb-b828-f4fa2d2356e6 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:33.625757 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c597cca-da38-48cb-b828-f4fa2d2356e6 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:33.627271 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c597cca-da38-48cb-b828-f4fa2d2356e6 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:33.629539 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 877/1720] 10.222.0.25 () {54 vars in 921 bytes} [Thu Aug 4 21:25:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:25:33.630089 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecbeac4-a2c5-433b-a91c-cca3c8e963c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:33.630696 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecbeac4-a2c5-433b-a91c-cca3c8e963c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:33.632069 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecbeac4-a2c5-433b-a91c-cca3c8e963c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:33.633232 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecbeac4-a2c5-433b-a91c-cca3c8e963c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:33.640340 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda88be0-a45a-4f95-b26a-496102fd9972 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:33.641010 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda88be0-a45a-4f95-b26a-496102fd9972 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:33.641578 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda88be0-a45a-4f95-b26a-496102fd9972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:33.656812 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecbeac4-a2c5-433b-a91c-cca3c8e963c6 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:33.734134 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-cda88be0-a45a-4f95-b26a-496102fd9972 None None] MFA Rules not processed for user `6a2ce1df87fa47e5be301916bf3245fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:25:33.735743 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-cda88be0-a45a-4f95-b26a-496102fd9972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:33.866844 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 878/1721] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Aug 4 21:25:33 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:25:34.039255 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 844/1722] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:25:33 2022] POST /identity/v3/projects => generated 375 bytes in 520 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:25:34.040464 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda88be0-a45a-4f95-b26a-496102fd9972 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:34.071165 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda88be0-a45a-4f95-b26a-496102fd9972 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:34.308169 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8bb61a-13e9-4101-9807-4ecfe9b86d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:34.308911 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8bb61a-13e9-4101-9807-4ecfe9b86d0a admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:34.309199 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8bb61a-13e9-4101-9807-4ecfe9b86d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:34.309632 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8bb61a-13e9-4101-9807-4ecfe9b86d0a admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:34.310416 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8bb61a-13e9-4101-9807-4ecfe9b86d0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:34.312980 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8bb61a-13e9-4101-9807-4ecfe9b86d0a admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:34.456353 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 879/1723] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:25:34 2022] POST /identity/v3/users => generated 288 bytes in 416 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:25:34.460652 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecbeac4-a2c5-433b-a91c-cca3c8e963c6 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:34.524700 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecbeac4-a2c5-433b-a91c-cca3c8e963c6 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:34.580519 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c2f9b6-4756-41bb-ae5d-2a6ffde12769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:34.584854 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c2f9b6-4756-41bb-ae5d-2a6ffde12769 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:34.585408 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c2f9b6-4756-41bb-ae5d-2a6ffde12769 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:34.627272 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c2f9b6-4756-41bb-ae5d-2a6ffde12769 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:34.627272 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c2f9b6-4756-41bb-ae5d-2a6ffde12769 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:34.627272 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c2f9b6-4756-41bb-ae5d-2a6ffde12769 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:34.627272 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 845/1724] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:25:34 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:25:34.653817 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8bb61a-13e9-4101-9807-4ecfe9b86d0a admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:34.653817 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8bb61a-13e9-4101-9807-4ecfe9b86d0a admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:34.670587 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd293fb1-a88b-4c59-a37f-310c96ad572b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:34.671245 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd293fb1-a88b-4c59-a37f-310c96ad572b admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:34.671694 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd293fb1-a88b-4c59-a37f-310c96ad572b admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:34.671859 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd293fb1-a88b-4c59-a37f-310c96ad572b admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:34.673524 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd293fb1-a88b-4c59-a37f-310c96ad572b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:34.677487 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd293fb1-a88b-4c59-a37f-310c96ad572b admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:34.742535 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c2f9b6-4756-41bb-ae5d-2a6ffde12769 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:34.753810 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 880/1725] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:25:34 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:25:34.761017 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c2f9b6-4756-41bb-ae5d-2a6ffde12769 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:34.794040 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376b00ed-fa9b-4f58-b88e-ca8756b9434c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:34.794935 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376b00ed-fa9b-4f58-b88e-ca8756b9434c admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:34.795514 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376b00ed-fa9b-4f58-b88e-ca8756b9434c admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:34.795900 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376b00ed-fa9b-4f58-b88e-ca8756b9434c admin admin] PATH_INFO: `/v3/projects/9509abcec8af4bba8056aa0e713a5370/users/b7b767fcc40a4aa5b10aef447745a977/roles/8feebc60363c4feb870718b1721e5324` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:34.859049 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376b00ed-fa9b-4f58-b88e-ca8756b9434c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9509abcec8af4bba8056aa0e713a5370, user_id=b7b767fcc40a4aa5b10aef447745a977, role_id=8feebc60363c4feb870718b1721e5324)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:34.874399 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376b00ed-fa9b-4f58-b88e-ca8756b9434c admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:35.638531 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 846/1726] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:25:34 2022] PUT /identity/v3/projects/9509abcec8af4bba8056aa0e713a5370/users/b7b767fcc40a4aa5b10aef447745a977/roles/8feebc60363c4feb870718b1721e5324 => generated 0 bytes in 774 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:25:35.640004 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcf7aaf-e237-4b5e-94a1-77dbece3c2b3 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:35.640004 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcf7aaf-e237-4b5e-94a1-77dbece3c2b3 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:35.640004 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcf7aaf-e237-4b5e-94a1-77dbece3c2b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:35.716644 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376b00ed-fa9b-4f58-b88e-ca8756b9434c admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:35.717332 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-376b00ed-fa9b-4f58-b88e-ca8756b9434c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:35.732134 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-afcf7aaf-e237-4b5e-94a1-77dbece3c2b3 None None] MFA Rules not processed for user `b7b767fcc40a4aa5b10aef447745a977`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:25:35.734846 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-afcf7aaf-e237-4b5e-94a1-77dbece3c2b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:35.847262 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:35.970507 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376b00ed-fa9b-4f58-b88e-ca8756b9434c admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:35.989919 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e109da-a3db-47c8-b856-b1a81c286c1b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a2ce1df87fa47e5be301916bf3245fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:35.990579 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e109da-a3db-47c8-b856-b1a81c286c1b service nova] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:35.990864 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e109da-a3db-47c8-b856-b1a81c286c1b service nova] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:35.991148 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e109da-a3db-47c8-b856-b1a81c286c1b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:36.019946 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 881/1727] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:25:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 386 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:25:36.036969 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcf7aaf-e237-4b5e-94a1-77dbece3c2b3 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:36.037263 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e109da-a3db-47c8-b856-b1a81c286c1b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:36.037263 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e109da-a3db-47c8-b856-b1a81c286c1b service nova] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:36.037459 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-afcf7aaf-e237-4b5e-94a1-77dbece3c2b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:36.048497 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 847/1728] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:25:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:25:36.369596 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcf7aaf-e237-4b5e-94a1-77dbece3c2b3 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:36.386457 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab1a372-6226-4b01-9b91-45623c7f8718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:36.388614 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab1a372-6226-4b01-9b91-45623c7f8718 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:36.388900 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab1a372-6226-4b01-9b91-45623c7f8718 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:36.392923 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab1a372-6226-4b01-9b91-45623c7f8718 admin admin] PATH_INFO: `/v3/users/a92fe8a407de490fb24d4870c234b43d` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:36.395098 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab1a372-6226-4b01-9b91-45623c7f8718 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a92fe8a407de490fb24d4870c234b43d)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:36.402917 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab1a372-6226-4b01-9b91-45623c7f8718 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:37.014108 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-7ab1a372-6226-4b01-9b91-45623c7f8718 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a92fe8a407de490fb24d4870c234b43d'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:37.089653 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-7ab1a372-6226-4b01-9b91-45623c7f8718 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a92fe8a407de490fb24d4870c234b43d'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:37.092847 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-7ab1a372-6226-4b01-9b91-45623c7f8718 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a92fe8a407de490fb24d4870c234b43d'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:37.095914 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-7ab1a372-6226-4b01-9b91-45623c7f8718 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a92fe8a407de490fb24d4870c234b43d'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:37.100384 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e109da-a3db-47c8-b856-b1a81c286c1b service nova] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:37.104492 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-70e109da-a3db-47c8-b856-b1a81c286c1b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:37.169894 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e109da-a3db-47c8-b856-b1a81c286c1b service nova] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:37.254332 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f0786c-3c73-4091-9f5c-48fe0d553106 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a2ce1df87fa47e5be301916bf3245fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:37.254332 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f0786c-3c73-4091-9f5c-48fe0d553106 service nova] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:37.254332 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f0786c-3c73-4091-9f5c-48fe0d553106 service nova] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:37.254332 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f0786c-3c73-4091-9f5c-48fe0d553106 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:37.254866 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-27f0786c-3c73-4091-9f5c-48fe0d553106 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:37.277345 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f0786c-3c73-4091-9f5c-48fe0d553106 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:37.279156 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f0786c-3c73-4091-9f5c-48fe0d553106 service nova] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:37.310677 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 848/1729] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:25:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:25:37.346853 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-7ab1a372-6226-4b01-9b91-45623c7f8718 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a92fe8a407de490fb24d4870c234b43d'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:37.420055 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 882/1730] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:25:36 2022] DELETE /identity/v3/users/a92fe8a407de490fb24d4870c234b43d => generated 0 bytes in 1384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:25:37.421831 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78153d9-f4ac-40a5-8ed6-ebb9fe6f84e6 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:37.422237 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78153d9-f4ac-40a5-8ed6-ebb9fe6f84e6 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:37.422770 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78153d9-f4ac-40a5-8ed6-ebb9fe6f84e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:37.484242 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-e78153d9-f4ac-40a5-8ed6-ebb9fe6f84e6 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:25:37.486945 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-e78153d9-f4ac-40a5-8ed6-ebb9fe6f84e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:37.639889 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:37.805482 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 849/1731] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:25:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:25:37.831798 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab1a372-6226-4b01-9b91-45623c7f8718 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:37.836033 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-7ab1a372-6226-4b01-9b91-45623c7f8718 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:37.959343 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab1a372-6226-4b01-9b91-45623c7f8718 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:37.968049 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65133d5a-2908-4c68-9821-5cbea81226c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:37.968049 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65133d5a-2908-4c68-9821-5cbea81226c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:37.968049 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65133d5a-2908-4c68-9821-5cbea81226c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:37.968049 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65133d5a-2908-4c68-9821-5cbea81226c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:37.981498 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65133d5a-2908-4c68-9821-5cbea81226c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:37.983065 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65133d5a-2908-4c68-9821-5cbea81226c4 service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:38.008268 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 883/1732] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:25:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:25:38.098062 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8216bda-aff5-413a-a717-0772e12a56f3 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:38.098361 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8216bda-aff5-413a-a717-0772e12a56f3 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:38.098646 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8216bda-aff5-413a-a717-0772e12a56f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:38.145984 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-a8216bda-aff5-413a-a717-0772e12a56f3 None None] MFA Rules not processed for user `9b29c3f2c29d4e2aba552c144cff94f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:25:38.147339 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-a8216bda-aff5-413a-a717-0772e12a56f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:38.289690 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 850/1733] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:25:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:25:38.951930 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb6fa73-bf44-42d4-9e05-8c548d47d082 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:38.952534 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb6fa73-bf44-42d4-9e05-8c548d47d082 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:38.952626 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb6fa73-bf44-42d4-9e05-8c548d47d082 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:38.954495 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 884/1734] 10.222.0.25 () {54 vars in 921 bytes} [Thu Aug 4 21:25:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:25:38.969064 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0088d909-06e8-420d-8da2-530915055908 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:38.969361 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0088d909-06e8-420d-8da2-530915055908 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:38.970227 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0088d909-06e8-420d-8da2-530915055908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:39.020509 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-0088d909-06e8-420d-8da2-530915055908 None None] MFA Rules not processed for user `6a2ce1df87fa47e5be301916bf3245fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:25:39.023968 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb6fa73-bf44-42d4-9e05-8c548d47d082 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:39.025925 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-afb6fa73-bf44-42d4-9e05-8c548d47d082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:39.028091 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-0088d909-06e8-420d-8da2-530915055908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:39.072221 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb6fa73-bf44-42d4-9e05-8c548d47d082 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:39.085840 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e9f2cb-8d08-415e-91d8-bb27feac391b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:39.089150 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e9f2cb-8d08-415e-91d8-bb27feac391b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:39.089150 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e9f2cb-8d08-415e-91d8-bb27feac391b admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:39.089150 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e9f2cb-8d08-415e-91d8-bb27feac391b admin admin] PATH_INFO: `/v3/users/b7b767fcc40a4aa5b10aef447745a977` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:39.096444 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e9f2cb-8d08-415e-91d8-bb27feac391b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7b767fcc40a4aa5b10aef447745a977)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:39.098622 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e9f2cb-8d08-415e-91d8-bb27feac391b admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:39.286939 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 851/1735] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Aug 4 21:25:38 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:25:39.510295 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-a4e9f2cb-8d08-415e-91d8-bb27feac391b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7b767fcc40a4aa5b10aef447745a977'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:39.544803 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-a4e9f2cb-8d08-415e-91d8-bb27feac391b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7b767fcc40a4aa5b10aef447745a977'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:39.545911 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-a4e9f2cb-8d08-415e-91d8-bb27feac391b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7b767fcc40a4aa5b10aef447745a977'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:39.547396 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-a4e9f2cb-8d08-415e-91d8-bb27feac391b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7b767fcc40a4aa5b10aef447745a977'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:39.839073 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0088d909-06e8-420d-8da2-530915055908 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:39.840564 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-0088d909-06e8-420d-8da2-530915055908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:39.875417 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0088d909-06e8-420d-8da2-530915055908 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:39.883657 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c36790-8040-47a1-86ef-97f063749edc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a2ce1df87fa47e5be301916bf3245fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:39.885896 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c36790-8040-47a1-86ef-97f063749edc service nova] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:39.887711 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c36790-8040-47a1-86ef-97f063749edc service nova] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:39.889043 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c36790-8040-47a1-86ef-97f063749edc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:39.896046 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-a4c36790-8040-47a1-86ef-97f063749edc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:39.925581 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c36790-8040-47a1-86ef-97f063749edc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:39.930651 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c36790-8040-47a1-86ef-97f063749edc service nova] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:39.951891 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 852/1736] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:25:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:25:40.177986 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-a4e9f2cb-8d08-415e-91d8-bb27feac391b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7b767fcc40a4aa5b10aef447745a977'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:40.202011 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 885/1737] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:25:39 2022] DELETE /identity/v3/users/b7b767fcc40a4aa5b10aef447745a977 => generated 0 bytes in 1179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:25:40.207643 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195d40cd-d625-4b6d-9b72-7724adb72373 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:40.208252 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195d40cd-d625-4b6d-9b72-7724adb72373 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:40.208568 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195d40cd-d625-4b6d-9b72-7724adb72373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:40.300742 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-195d40cd-d625-4b6d-9b72-7724adb72373 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:25:40.314398 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-195d40cd-d625-4b6d-9b72-7724adb72373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:40.482237 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 853/1738] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:25:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:25:40.599645 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e9f2cb-8d08-415e-91d8-bb27feac391b admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:40.601644 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-a4e9f2cb-8d08-415e-91d8-bb27feac391b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:40.725467 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e9f2cb-8d08-415e-91d8-bb27feac391b admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:40.730603 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23408ae-dd71-4bea-bef9-7853a0cd42ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:40.731489 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23408ae-dd71-4bea-bef9-7853a0cd42ac service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:40.732052 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23408ae-dd71-4bea-bef9-7853a0cd42ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:40.732333 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23408ae-dd71-4bea-bef9-7853a0cd42ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:40.745740 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23408ae-dd71-4bea-bef9-7853a0cd42ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:40.747235 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23408ae-dd71-4bea-bef9-7853a0cd42ac service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:40.759703 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 886/1739] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:25:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:25:41.119481 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195d40cd-d625-4b6d-9b72-7724adb72373 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:41.122609 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-195d40cd-d625-4b6d-9b72-7724adb72373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:41.139833 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195d40cd-d625-4b6d-9b72-7724adb72373 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:41.144915 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7f88b0-d3b8-41a3-bd7c-8a9e51d4f98a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:41.145587 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7f88b0-d3b8-41a3-bd7c-8a9e51d4f98a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:41.146051 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7f88b0-d3b8-41a3-bd7c-8a9e51d4f98a admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:41.146485 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7f88b0-d3b8-41a3-bd7c-8a9e51d4f98a admin admin] PATH_INFO: `/v3/projects/ecb46496afcd4f3fb0f689c172a6603e` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:41.148287 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7f88b0-d3b8-41a3-bd7c-8a9e51d4f98a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ecb46496afcd4f3fb0f689c172a6603e)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:41.150657 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7f88b0-d3b8-41a3-bd7c-8a9e51d4f98a admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:41.325387 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ed7f88b0-d3b8-41a3-bd7c-8a9e51d4f98a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ecb46496afcd4f3fb0f689c172a6603e', 'request_id': 'req-ed7f88b0-d3b8-41a3-bd7c-8a9e51d4f98a'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:41.334678 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ed7f88b0-d3b8-41a3-bd7c-8a9e51d4f98a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ecb46496afcd4f3fb0f689c172a6603e', 'request_id': 'req-ed7f88b0-d3b8-41a3-bd7c-8a9e51d4f98a'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:41.449493 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ed7f88b0-d3b8-41a3-bd7c-8a9e51d4f98a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ecb46496afcd4f3fb0f689c172a6603e', 'request_id': 'req-ed7f88b0-d3b8-41a3-bd7c-8a9e51d4f98a'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:41.465251 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ed7f88b0-d3b8-41a3-bd7c-8a9e51d4f98a admin admin] The token cache is being invalidate because project ecb46496afcd4f3fb0f689c172a6603e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:25:41.465625 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ed7f88b0-d3b8-41a3-bd7c-8a9e51d4f98a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:41.466637 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ed7f88b0-d3b8-41a3-bd7c-8a9e51d4f98a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:41.469063 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 854/1740] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:25:41 2022] DELETE /identity/v3/projects/ecb46496afcd4f3fb0f689c172a6603e => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:25:41.473480 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23408ae-dd71-4bea-bef9-7853a0cd42ac service neutron] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:41.490239 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23408ae-dd71-4bea-bef9-7853a0cd42ac service neutron] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:41.559619 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8806fe7-c143-468f-bd32-87bfad737017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:41.560529 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8806fe7-c143-468f-bd32-87bfad737017 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:41.560978 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8806fe7-c143-468f-bd32-87bfad737017 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:41.561410 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8806fe7-c143-468f-bd32-87bfad737017 admin admin] PATH_INFO: `/v3/users/4bedebd0e83147e4922a4f85bc6e3536` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:41.563304 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8806fe7-c143-468f-bd32-87bfad737017 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4bedebd0e83147e4922a4f85bc6e3536)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:41.565827 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8806fe7-c143-468f-bd32-87bfad737017 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:41.655405 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7f88b0-d3b8-41a3-bd7c-8a9e51d4f98a admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:41.657527 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-ed7f88b0-d3b8-41a3-bd7c-8a9e51d4f98a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:41.675711 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7f88b0-d3b8-41a3-bd7c-8a9e51d4f98a admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:41.683265 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2242d36-fea5-4f8a-9d0b-18cf407db8fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:41.683832 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2242d36-fea5-4f8a-9d0b-18cf407db8fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:41.684119 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2242d36-fea5-4f8a-9d0b-18cf407db8fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:41.684587 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2242d36-fea5-4f8a-9d0b-18cf407db8fa admin admin] PATH_INFO: `/v3/projects/9509abcec8af4bba8056aa0e713a5370` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:41.686291 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2242d36-fea5-4f8a-9d0b-18cf407db8fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=9509abcec8af4bba8056aa0e713a5370)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:41.689828 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2242d36-fea5-4f8a-9d0b-18cf407db8fa admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:41.805477 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-c8806fe7-c143-468f-bd32-87bfad737017 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4bedebd0e83147e4922a4f85bc6e3536'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:41.839864 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-c8806fe7-c143-468f-bd32-87bfad737017 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4bedebd0e83147e4922a4f85bc6e3536'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:41.840633 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-c8806fe7-c143-468f-bd32-87bfad737017 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4bedebd0e83147e4922a4f85bc6e3536'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:41.841248 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-c8806fe7-c143-468f-bd32-87bfad737017 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4bedebd0e83147e4922a4f85bc6e3536'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:41.853777 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c2242d36-fea5-4f8a-9d0b-18cf407db8fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9509abcec8af4bba8056aa0e713a5370', 'request_id': 'req-c2242d36-fea5-4f8a-9d0b-18cf407db8fa'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:41.868706 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c2242d36-fea5-4f8a-9d0b-18cf407db8fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9509abcec8af4bba8056aa0e713a5370', 'request_id': 'req-c2242d36-fea5-4f8a-9d0b-18cf407db8fa'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:41.948589 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-c8806fe7-c143-468f-bd32-87bfad737017 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4bedebd0e83147e4922a4f85bc6e3536'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:41.964928 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 887/1741] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:25:41 2022] DELETE /identity/v3/users/4bedebd0e83147e4922a4f85bc6e3536 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:25:42.051034 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c2242d36-fea5-4f8a-9d0b-18cf407db8fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9509abcec8af4bba8056aa0e713a5370', 'request_id': 'req-c2242d36-fea5-4f8a-9d0b-18cf407db8fa'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:42.108580 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c2242d36-fea5-4f8a-9d0b-18cf407db8fa admin admin] The token cache is being invalidate because project 9509abcec8af4bba8056aa0e713a5370 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:25:42.109526 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c2242d36-fea5-4f8a-9d0b-18cf407db8fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:42.116366 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c2242d36-fea5-4f8a-9d0b-18cf407db8fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:42.120229 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 855/1742] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:25:41 2022] DELETE /identity/v3/projects/9509abcec8af4bba8056aa0e713a5370 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:25:42.125916 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bff00f-356b-4388-896d-d5ef80caed73 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:42.126986 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bff00f-356b-4388-896d-d5ef80caed73 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:42.126986 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bff00f-356b-4388-896d-d5ef80caed73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:42.144498 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:42.172319 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-35bff00f-356b-4388-896d-d5ef80caed73 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:25:42.173672 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-35bff00f-356b-4388-896d-d5ef80caed73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:42.363995 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 888/1743] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:25:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:25:42.372510 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68649f44-5c28-4ecb-93e3-b3cb3f75a64b None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:42.373486 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68649f44-5c28-4ecb-93e3-b3cb3f75a64b None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:42.373486 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68649f44-5c28-4ecb-93e3-b3cb3f75a64b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:42.397710 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-68649f44-5c28-4ecb-93e3-b3cb3f75a64b None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:25:42.401112 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-68649f44-5c28-4ecb-93e3-b3cb3f75a64b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:42.408556 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 856/1744] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:25:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:25:42.412788 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bff00f-356b-4388-896d-d5ef80caed73 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:42.425362 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bff00f-356b-4388-896d-d5ef80caed73 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:42.430774 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea4a409-b8fe-4231-85f7-aa2c60d4014c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:42.431659 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea4a409-b8fe-4231-85f7-aa2c60d4014c admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:42.432097 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea4a409-b8fe-4231-85f7-aa2c60d4014c admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:42.432532 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea4a409-b8fe-4231-85f7-aa2c60d4014c admin admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:42.433403 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea4a409-b8fe-4231-85f7-aa2c60d4014c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:25:42.434201 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea4a409-b8fe-4231-85f7-aa2c60d4014c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:42.436110 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea4a409-b8fe-4231-85f7-aa2c60d4014c admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:42.459422 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68649f44-5c28-4ecb-93e3-b3cb3f75a64b None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:42.460740 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 889/1745] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Aug 4 21:25:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:25:42.485520 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68649f44-5c28-4ecb-93e3-b3cb3f75a64b None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:42.491140 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c20af95-261d-4a60-b5b0-b6b4c8c874cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:42.491828 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c20af95-261d-4a60-b5b0-b6b4c8c874cc admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:42.492280 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c20af95-261d-4a60-b5b0-b6b4c8c874cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:42.492778 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c20af95-261d-4a60-b5b0-b6b4c8c874cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:42.493689 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c20af95-261d-4a60-b5b0-b6b4c8c874cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:42.496323 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c20af95-261d-4a60-b5b0-b6b4c8c874cc admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:42.552382 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:42.642200 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 857/1746] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:25:42 2022] POST /identity/v3/projects => generated 381 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:25:42.652198 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea4a409-b8fe-4231-85f7-aa2c60d4014c admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:42.662490 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea4a409-b8fe-4231-85f7-aa2c60d4014c admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:42.684991 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c20af95-261d-4a60-b5b0-b6b4c8c874cc admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:42.689479 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-0c20af95-261d-4a60-b5b0-b6b4c8c874cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:42.774485 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ad13ad-dbe4-4e12-8f72-8a1f62b852cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:42.778163 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ad13ad-dbe4-4e12-8f72-8a1f62b852cd admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:42.778709 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ad13ad-dbe4-4e12-8f72-8a1f62b852cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:42.779098 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ad13ad-dbe4-4e12-8f72-8a1f62b852cd admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:42.779750 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c20af95-261d-4a60-b5b0-b6b4c8c874cc admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:42.779988 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ad13ad-dbe4-4e12-8f72-8a1f62b852cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:42.782997 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ad13ad-dbe4-4e12-8f72-8a1f62b852cd admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:42.786918 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13bca94-b2c0-45db-a65f-75d4d3959532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:42.787733 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13bca94-b2c0-45db-a65f-75d4d3959532 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:42.788148 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13bca94-b2c0-45db-a65f-75d4d3959532 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:42.788495 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13bca94-b2c0-45db-a65f-75d4d3959532 admin admin] PATH_INFO: `/v3/projects/51b088994e4b40c5b54627cfbf7ae353` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:42.790439 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13bca94-b2c0-45db-a65f-75d4d3959532 admin admin] RBAC: Authorizing `identity:delete_project(project_id=51b088994e4b40c5b54627cfbf7ae353)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:42.794081 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13bca94-b2c0-45db-a65f-75d4d3959532 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:42.892598 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 890/1747] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:25:42 2022] POST /identity/v3/users => generated 291 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:25:42.896646 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ad13ad-dbe4-4e12-8f72-8a1f62b852cd admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:42.920638 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ad13ad-dbe4-4e12-8f72-8a1f62b852cd admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:42.944489 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-e13bca94-b2c0-45db-a65f-75d4d3959532 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51b088994e4b40c5b54627cfbf7ae353', 'request_id': 'req-e13bca94-b2c0-45db-a65f-75d4d3959532'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:42.953865 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-e13bca94-b2c0-45db-a65f-75d4d3959532 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51b088994e4b40c5b54627cfbf7ae353', 'request_id': 'req-e13bca94-b2c0-45db-a65f-75d4d3959532'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:43.009899 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9316c9e1-f280-4312-adc1-61eead716a65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:43.010873 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9316c9e1-f280-4312-adc1-61eead716a65 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:43.011288 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9316c9e1-f280-4312-adc1-61eead716a65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:43.011697 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9316c9e1-f280-4312-adc1-61eead716a65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:43.012628 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9316c9e1-f280-4312-adc1-61eead716a65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:43.014600 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9316c9e1-f280-4312-adc1-61eead716a65 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:43.030528 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 891/1748] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:25:42 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:25:43.034163 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9316c9e1-f280-4312-adc1-61eead716a65 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:43.051601 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9316c9e1-f280-4312-adc1-61eead716a65 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:43.058824 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e569413c-3492-445b-8d66-2d36b9c746b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:43.059743 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e569413c-3492-445b-8d66-2d36b9c746b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:43.060332 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e569413c-3492-445b-8d66-2d36b9c746b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:43.060596 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e569413c-3492-445b-8d66-2d36b9c746b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:43.061290 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e569413c-3492-445b-8d66-2d36b9c746b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:43.064550 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e569413c-3492-445b-8d66-2d36b9c746b3 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:43.078491 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 892/1749] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:25:43 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:25:43.082822 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e569413c-3492-445b-8d66-2d36b9c746b3 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:43.107658 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e569413c-3492-445b-8d66-2d36b9c746b3 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:43.107658 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a61aaee-74ed-4923-acc4-5877b8b8d923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:43.107658 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a61aaee-74ed-4923-acc4-5877b8b8d923 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:43.107658 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a61aaee-74ed-4923-acc4-5877b8b8d923 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:43.108049 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a61aaee-74ed-4923-acc4-5877b8b8d923 admin admin] PATH_INFO: `/v3/projects/ea1fd140c0414501bb7765130c4c0651/users/837f72c758964b2eb11786cd115baf5f/roles/8feebc60363c4feb870718b1721e5324` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:43.111555 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-e13bca94-b2c0-45db-a65f-75d4d3959532 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51b088994e4b40c5b54627cfbf7ae353', 'request_id': 'req-e13bca94-b2c0-45db-a65f-75d4d3959532'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:43.126954 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-e13bca94-b2c0-45db-a65f-75d4d3959532 admin admin] The token cache is being invalidate because project 51b088994e4b40c5b54627cfbf7ae353 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:25:43.127523 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-e13bca94-b2c0-45db-a65f-75d4d3959532 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:43.129148 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-e13bca94-b2c0-45db-a65f-75d4d3959532 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:43.131251 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 858/1750] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:25:42 2022] DELETE /identity/v3/projects/51b088994e4b40c5b54627cfbf7ae353 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:25:43.137107 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f001b9d-d81d-4635-b8ed-7a40431352ed None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:43.137673 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f001b9d-d81d-4635-b8ed-7a40431352ed None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:43.138153 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f001b9d-d81d-4635-b8ed-7a40431352ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:43.155082 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a61aaee-74ed-4923-acc4-5877b8b8d923 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea1fd140c0414501bb7765130c4c0651, user_id=837f72c758964b2eb11786cd115baf5f, role_id=8feebc60363c4feb870718b1721e5324)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:43.159177 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a61aaee-74ed-4923-acc4-5877b8b8d923 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:43.178540 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-4f001b9d-d81d-4635-b8ed-7a40431352ed None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:25:43.178540 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-4f001b9d-d81d-4635-b8ed-7a40431352ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:43.204546 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 859/1751] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:25:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:25:43.219017 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6325baf8-72f2-4af6-8afa-e4f65d4d3c7c None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:43.220718 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6325baf8-72f2-4af6-8afa-e4f65d4d3c7c None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:43.221005 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6325baf8-72f2-4af6-8afa-e4f65d4d3c7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:43.245990 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 893/1752] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:25:43 2022] PUT /identity/v3/projects/ea1fd140c0414501bb7765130c4c0651/users/837f72c758964b2eb11786cd115baf5f/roles/8feebc60363c4feb870718b1721e5324 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:25:43.262361 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7cec51-52f7-48d7-9843-75a485b5c2d9 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:43.262667 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7cec51-52f7-48d7-9843-75a485b5c2d9 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:43.262956 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7cec51-52f7-48d7-9843-75a485b5c2d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:43.269490 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-6325baf8-72f2-4af6-8afa-e4f65d4d3c7c None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:25:43.281604 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-6325baf8-72f2-4af6-8afa-e4f65d4d3c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:43.306642 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-4e7cec51-52f7-48d7-9843-75a485b5c2d9 None None] MFA Rules not processed for user `837f72c758964b2eb11786cd115baf5f`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:25:43.308126 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-4e7cec51-52f7-48d7-9843-75a485b5c2d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:43.394265 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 860/1753] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:25:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:25:43.398787 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6325baf8-72f2-4af6-8afa-e4f65d4d3c7c None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:43.421836 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 894/1754] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:25:43 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:25:43.422695 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6325baf8-72f2-4af6-8afa-e4f65d4d3c7c None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:43.424666 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7cec51-52f7-48d7-9843-75a485b5c2d9 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:43.426671 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-4e7cec51-52f7-48d7-9843-75a485b5c2d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:43.429539 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38fc3ec-7ce2-40f7-889a-fdb19673c2ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:43.430187 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38fc3ec-7ce2-40f7-889a-fdb19673c2ec admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:43.430475 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38fc3ec-7ce2-40f7-889a-fdb19673c2ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:43.430753 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38fc3ec-7ce2-40f7-889a-fdb19673c2ec admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:43.431668 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38fc3ec-7ce2-40f7-889a-fdb19673c2ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:25:43.432099 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38fc3ec-7ce2-40f7-889a-fdb19673c2ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:43.434000 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38fc3ec-7ce2-40f7-889a-fdb19673c2ec admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:43.452719 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 861/1755] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Aug 4 21:25:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:25:43.459533 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38fc3ec-7ce2-40f7-889a-fdb19673c2ec admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:43.474193 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38fc3ec-7ce2-40f7-889a-fdb19673c2ec admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:43.483936 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7cec51-52f7-48d7-9843-75a485b5c2d9 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:43.489414 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552f678f-1cec-4a9d-9c05-1bffe7e450f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:43.491199 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552f678f-1cec-4a9d-9c05-1bffe7e450f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:43.493041 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552f678f-1cec-4a9d-9c05-1bffe7e450f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:43.493530 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552f678f-1cec-4a9d-9c05-1bffe7e450f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:43.499443 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552f678f-1cec-4a9d-9c05-1bffe7e450f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:43.514022 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fe1b31-42d9-414a-aea4-df88ec001054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:43.514022 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fe1b31-42d9-414a-aea4-df88ec001054 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:43.514022 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fe1b31-42d9-414a-aea4-df88ec001054 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:43.514022 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fe1b31-42d9-414a-aea4-df88ec001054 admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:43.514550 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fe1b31-42d9-414a-aea4-df88ec001054 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:43.514639 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552f678f-1cec-4a9d-9c05-1bffe7e450f8 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:43.514711 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fe1b31-42d9-414a-aea4-df88ec001054 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:43.684971 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:43.906289 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 895/1756] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:25:43 2022] POST /identity/v3/projects => generated 377 bytes in 484 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:25:43.913304 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fe1b31-42d9-414a-aea4-df88ec001054 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:43.913943 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 862/1757] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:25:43 2022] POST /identity/v3/projects => generated 365 bytes in 455 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:25:43.918431 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552f678f-1cec-4a9d-9c05-1bffe7e450f8 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:44.038506 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fe1b31-42d9-414a-aea4-df88ec001054 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:44.053717 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552f678f-1cec-4a9d-9c05-1bffe7e450f8 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:44.062442 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ad835d-99cf-4da5-a0bd-260f4a6be610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:44.063937 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ad835d-99cf-4da5-a0bd-260f4a6be610 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:44.064827 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ad835d-99cf-4da5-a0bd-260f4a6be610 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:44.064962 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ad835d-99cf-4da5-a0bd-260f4a6be610 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:44.069912 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ad835d-99cf-4da5-a0bd-260f4a6be610 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:44.069912 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ad835d-99cf-4da5-a0bd-260f4a6be610 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:44.124781 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1e76ab-e67d-46a0-829d-f5897798a0c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:44.125430 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1e76ab-e67d-46a0-829d-f5897798a0c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:44.125779 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1e76ab-e67d-46a0-829d-f5897798a0c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:44.126567 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1e76ab-e67d-46a0-829d-f5897798a0c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:44.127218 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1e76ab-e67d-46a0-829d-f5897798a0c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:44.129294 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1e76ab-e67d-46a0-829d-f5897798a0c0 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:44.215304 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 863/1758] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:25:43 2022] POST /identity/v3/users => generated 283 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:25:44.216296 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ad835d-99cf-4da5-a0bd-260f4a6be610 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:44.232901 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ad835d-99cf-4da5-a0bd-260f4a6be610 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:44.246256 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd6fc54-169c-4346-a523-81bf6044874f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:44.246256 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd6fc54-169c-4346-a523-81bf6044874f admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:44.246256 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd6fc54-169c-4346-a523-81bf6044874f admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:44.246256 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd6fc54-169c-4346-a523-81bf6044874f admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:44.261314 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd6fc54-169c-4346-a523-81bf6044874f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:44.261314 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd6fc54-169c-4346-a523-81bf6044874f admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:44.261899 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 896/1759] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:25:43 2022] POST /identity/v3/users => generated 295 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:25:44.263725 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1e76ab-e67d-46a0-829d-f5897798a0c0 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:44.270924 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 864/1760] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:25:44 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:25:44.275553 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd6fc54-169c-4346-a523-81bf6044874f admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:44.278696 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1e76ab-e67d-46a0-829d-f5897798a0c0 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:44.287939 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0630d28-cef8-4231-89ef-4cfca01b76c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:44.288927 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0630d28-cef8-4231-89ef-4cfca01b76c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:44.289846 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0630d28-cef8-4231-89ef-4cfca01b76c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:44.290277 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0630d28-cef8-4231-89ef-4cfca01b76c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:44.291836 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0630d28-cef8-4231-89ef-4cfca01b76c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:44.293049 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd6fc54-169c-4346-a523-81bf6044874f admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:44.294768 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0630d28-cef8-4231-89ef-4cfca01b76c5 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:44.301041 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef945ee9-7671-4b73-af55-c0427f7f2a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:44.308817 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef945ee9-7671-4b73-af55-c0427f7f2a81 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:44.308817 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef945ee9-7671-4b73-af55-c0427f7f2a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:44.308817 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef945ee9-7671-4b73-af55-c0427f7f2a81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:44.308817 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef945ee9-7671-4b73-af55-c0427f7f2a81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:44.308817 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef945ee9-7671-4b73-af55-c0427f7f2a81 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:44.311932 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 897/1761] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:25:44 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:25:44.319763 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0630d28-cef8-4231-89ef-4cfca01b76c5 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:44.343740 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 865/1762] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:25:44 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:25:44.347266 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0630d28-cef8-4231-89ef-4cfca01b76c5 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:44.348386 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef945ee9-7671-4b73-af55-c0427f7f2a81 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:44.356156 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61f2245-90ff-414b-92b1-8950f77f34d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:44.361751 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61f2245-90ff-414b-92b1-8950f77f34d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:44.361751 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61f2245-90ff-414b-92b1-8950f77f34d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:44.361751 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61f2245-90ff-414b-92b1-8950f77f34d6 admin admin] PATH_INFO: `/v3/projects/06cd80fe6def42e49240cb05b4ff853f/users/a41964f0d1dd43a8b7b88dd8f68c7b24/roles/a432dd4735944ecfab2fa5ce1705d356` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:44.362020 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef945ee9-7671-4b73-af55-c0427f7f2a81 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:44.369696 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08bd393-0291-4d3a-bead-e8df551ee156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:44.370434 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08bd393-0291-4d3a-bead-e8df551ee156 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:44.370826 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08bd393-0291-4d3a-bead-e8df551ee156 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:44.371530 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08bd393-0291-4d3a-bead-e8df551ee156 admin admin] PATH_INFO: `/v3/projects/2aef5adc9b204616a7c18ac6cda47710/users/35505e9eab184580a2150babd6c33900/roles/8feebc60363c4feb870718b1721e5324` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:44.390102 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08bd393-0291-4d3a-bead-e8df551ee156 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2aef5adc9b204616a7c18ac6cda47710, user_id=35505e9eab184580a2150babd6c33900, role_id=8feebc60363c4feb870718b1721e5324)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:44.395055 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61f2245-90ff-414b-92b1-8950f77f34d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06cd80fe6def42e49240cb05b4ff853f, user_id=a41964f0d1dd43a8b7b88dd8f68c7b24, role_id=a432dd4735944ecfab2fa5ce1705d356)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:44.395055 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61f2245-90ff-414b-92b1-8950f77f34d6 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:44.395273 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08bd393-0291-4d3a-bead-e8df551ee156 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:44.492253 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 866/1763] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:25:44 2022] PUT /identity/v3/projects/2aef5adc9b204616a7c18ac6cda47710/users/35505e9eab184580a2150babd6c33900/roles/8feebc60363c4feb870718b1721e5324 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:25:44.494219 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 898/1764] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:25:44 2022] PUT /identity/v3/projects/06cd80fe6def42e49240cb05b4ff853f/users/a41964f0d1dd43a8b7b88dd8f68c7b24/roles/a432dd4735944ecfab2fa5ce1705d356 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:25:44.499414 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61f2245-90ff-414b-92b1-8950f77f34d6 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:44.499825 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d936e0-a978-4a87-a9e0-571dcaebace8 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:44.500319 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d936e0-a978-4a87-a9e0-571dcaebace8 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:44.500660 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d936e0-a978-4a87-a9e0-571dcaebace8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:44.566375 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-10d936e0-a978-4a87-a9e0-571dcaebace8 None None] MFA Rules not processed for user `35505e9eab184580a2150babd6c33900`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:25:44.567726 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-10d936e0-a978-4a87-a9e0-571dcaebace8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:44.629197 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61f2245-90ff-414b-92b1-8950f77f34d6 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:44.634709 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6befa27-dc6e-4517-b9db-6af434d6c16c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:44.635548 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6befa27-dc6e-4517-b9db-6af434d6c16c admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:44.635832 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6befa27-dc6e-4517-b9db-6af434d6c16c admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:44.636523 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6befa27-dc6e-4517-b9db-6af434d6c16c admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:44.636829 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6befa27-dc6e-4517-b9db-6af434d6c16c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:44.639576 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6befa27-dc6e-4517-b9db-6af434d6c16c admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:44.670269 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 899/1765] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:25:44 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:25:44.671656 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6befa27-dc6e-4517-b9db-6af434d6c16c admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:44.690011 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6befa27-dc6e-4517-b9db-6af434d6c16c admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:44.694420 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 867/1766] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:25:44 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:25:44.696141 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf457c80-3955-4b44-8328-9a2677f06b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:44.696893 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf457c80-3955-4b44-8328-9a2677f06b3c admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:44.697183 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf457c80-3955-4b44-8328-9a2677f06b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:44.697464 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf457c80-3955-4b44-8328-9a2677f06b3c admin admin] PATH_INFO: `/v3/projects/06cd80fe6def42e49240cb05b4ff853f/users/a41964f0d1dd43a8b7b88dd8f68c7b24/roles/a432dd4735944ecfab2fa5ce1705d356` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:44.699110 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d936e0-a978-4a87-a9e0-571dcaebace8 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:44.699814 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf457c80-3955-4b44-8328-9a2677f06b3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=06cd80fe6def42e49240cb05b4ff853f, user_id=a41964f0d1dd43a8b7b88dd8f68c7b24, role_id=a432dd4735944ecfab2fa5ce1705d356)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:44.704907 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf457c80-3955-4b44-8328-9a2677f06b3c admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:44.714509 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d936e0-a978-4a87-a9e0-571dcaebace8 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:44.721524 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 900/1767] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:25:44 2022] PUT /identity/v3/projects/06cd80fe6def42e49240cb05b4ff853f/users/a41964f0d1dd43a8b7b88dd8f68c7b24/roles/a432dd4735944ecfab2fa5ce1705d356 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:25:44.738373 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cd9404-93e1-4d13-87b0-89afe792692e None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:44.739131 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cd9404-93e1-4d13-87b0-89afe792692e None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:44.741288 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cd9404-93e1-4d13-87b0-89afe792692e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:44.775022 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-f7cd9404-93e1-4d13-87b0-89afe792692e None None] MFA Rules not processed for user `a41964f0d1dd43a8b7b88dd8f68c7b24`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:25:44.776284 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-f7cd9404-93e1-4d13-87b0-89afe792692e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:44.821942 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695ab5a8-2934-4e82-a1e7-938b76a1e4a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:44.822760 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695ab5a8-2934-4e82-a1e7-938b76a1e4a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:44.823048 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695ab5a8-2934-4e82-a1e7-938b76a1e4a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:44.823336 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695ab5a8-2934-4e82-a1e7-938b76a1e4a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:44.824047 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695ab5a8-2934-4e82-a1e7-938b76a1e4a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:44.826099 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695ab5a8-2934-4e82-a1e7-938b76a1e4a3 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:44.865600 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 901/1768] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:25:44 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:25:44.879041 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cd9404-93e1-4d13-87b0-89afe792692e None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:44.880580 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-f7cd9404-93e1-4d13-87b0-89afe792692e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:44.921771 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 868/1769] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:25:44 2022] POST /identity/v3/projects => generated 363 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:25:44.922662 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695ab5a8-2934-4e82-a1e7-938b76a1e4a3 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:44.959175 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cd9404-93e1-4d13-87b0-89afe792692e None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:45.004524 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695ab5a8-2934-4e82-a1e7-938b76a1e4a3 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:45.010955 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad96699d-f9ff-4521-a02f-737910f4094a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:45.019924 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad96699d-f9ff-4521-a02f-737910f4094a admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:45.019924 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad96699d-f9ff-4521-a02f-737910f4094a admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:45.019924 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad96699d-f9ff-4521-a02f-737910f4094a admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:45.019924 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad96699d-f9ff-4521-a02f-737910f4094a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:45.019924 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad96699d-f9ff-4521-a02f-737910f4094a admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:45.022887 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fffe59-b39c-4521-85f2-cca7096e92f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e31f2728056b460fa474478eb85ca1d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:45.023563 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fffe59-b39c-4521-85f2-cca7096e92f6 service cinder] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:45.023877 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fffe59-b39c-4521-85f2-cca7096e92f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:45.024404 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fffe59-b39c-4521-85f2-cca7096e92f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:45.096076 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fffe59-b39c-4521-85f2-cca7096e92f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:45.097553 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fffe59-b39c-4521-85f2-cca7096e92f6 service cinder] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:45.108203 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 902/1770] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:25:44 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:25:45.183410 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 869/1771] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:25:44 2022] POST /identity/v3/users => generated 288 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:25:45.184689 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fffe59-b39c-4521-85f2-cca7096e92f6 service cinder] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:45.199935 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fffe59-b39c-4521-85f2-cca7096e92f6 service cinder] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:45.205803 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8069f7af-7afd-44fc-8449-fd1cb4185619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:45.206546 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8069f7af-7afd-44fc-8449-fd1cb4185619 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:45.207162 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8069f7af-7afd-44fc-8449-fd1cb4185619 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:45.212427 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8069f7af-7afd-44fc-8449-fd1cb4185619 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:45.212427 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8069f7af-7afd-44fc-8449-fd1cb4185619 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:45.212427 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8069f7af-7afd-44fc-8449-fd1cb4185619 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:45.230677 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 903/1772] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:25:45 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:25:45.231933 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad96699d-f9ff-4521-a02f-737910f4094a admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:45.243812 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad96699d-f9ff-4521-a02f-737910f4094a admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:45.248881 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8ca97c-2aab-4215-8114-818a3785022b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:45.249575 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8ca97c-2aab-4215-8114-818a3785022b admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:45.249864 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8ca97c-2aab-4215-8114-818a3785022b admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:45.250126 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8ca97c-2aab-4215-8114-818a3785022b admin admin] PATH_INFO: `/v3/projects/db7d7ca21538415a979dffc032b6ddd1/users/0fe596f1c0d14f3f9e02e890f1bee050/roles/a432dd4735944ecfab2fa5ce1705d356` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:45.277967 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8ca97c-2aab-4215-8114-818a3785022b admin admin] RBAC: Authorizing `identity:create_grant(project_id=db7d7ca21538415a979dffc032b6ddd1, user_id=0fe596f1c0d14f3f9e02e890f1bee050, role_id=a432dd4735944ecfab2fa5ce1705d356)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:45.284169 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8ca97c-2aab-4215-8114-818a3785022b admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:45.397496 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 870/1773] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:25:45 2022] PUT /identity/v3/projects/db7d7ca21538415a979dffc032b6ddd1/users/0fe596f1c0d14f3f9e02e890f1bee050/roles/a432dd4735944ecfab2fa5ce1705d356 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:25:45.405535 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8069f7af-7afd-44fc-8449-fd1cb4185619 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:45.418061 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8069f7af-7afd-44fc-8449-fd1cb4185619 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:45.532930 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293604dc-291d-4f1b-a2d2-0a692da80bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:45.533969 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293604dc-291d-4f1b-a2d2-0a692da80bea admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:45.534584 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293604dc-291d-4f1b-a2d2-0a692da80bea admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:45.535070 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293604dc-291d-4f1b-a2d2-0a692da80bea admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:45.536200 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293604dc-291d-4f1b-a2d2-0a692da80bea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:45.538380 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293604dc-291d-4f1b-a2d2-0a692da80bea admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:45.555274 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 904/1774] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:25:45 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:25:45.560993 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8ca97c-2aab-4215-8114-818a3785022b admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:45.574059 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8ca97c-2aab-4215-8114-818a3785022b admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:45.581514 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccff1c0-da2b-4b3b-92af-4ab9bd0ca777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:45.582252 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccff1c0-da2b-4b3b-92af-4ab9bd0ca777 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:45.582538 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccff1c0-da2b-4b3b-92af-4ab9bd0ca777 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:45.582822 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccff1c0-da2b-4b3b-92af-4ab9bd0ca777 admin admin] PATH_INFO: `/v3/projects/db7d7ca21538415a979dffc032b6ddd1/users/0fe596f1c0d14f3f9e02e890f1bee050/roles/a432dd4735944ecfab2fa5ce1705d356` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:45.585534 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccff1c0-da2b-4b3b-92af-4ab9bd0ca777 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db7d7ca21538415a979dffc032b6ddd1, user_id=0fe596f1c0d14f3f9e02e890f1bee050, role_id=a432dd4735944ecfab2fa5ce1705d356)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:45.587780 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccff1c0-da2b-4b3b-92af-4ab9bd0ca777 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:45.601106 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 871/1775] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:25:45 2022] PUT /identity/v3/projects/db7d7ca21538415a979dffc032b6ddd1/users/0fe596f1c0d14f3f9e02e890f1bee050/roles/a432dd4735944ecfab2fa5ce1705d356 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:25:45.618329 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd3faf6-b87f-4a76-b833-7465afd0c4fb None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:45.618658 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd3faf6-b87f-4a76-b833-7465afd0c4fb None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:45.619025 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd3faf6-b87f-4a76-b833-7465afd0c4fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:45.656353 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-afd3faf6-b87f-4a76-b833-7465afd0c4fb None None] MFA Rules not processed for user `0fe596f1c0d14f3f9e02e890f1bee050`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:25:45.656864 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-afd3faf6-b87f-4a76-b833-7465afd0c4fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:45.733552 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 905/1776] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:25:45 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:25:45.745792 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccff1c0-da2b-4b3b-92af-4ab9bd0ca777 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:45.748186 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-0ccff1c0-da2b-4b3b-92af-4ab9bd0ca777 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:45.827855 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccff1c0-da2b-4b3b-92af-4ab9bd0ca777 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:45.832507 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a139959c-b6ba-42c0-8340-8fea54b733dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e31f2728056b460fa474478eb85ca1d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:45.833829 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a139959c-b6ba-42c0-8340-8fea54b733dc service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:45.834117 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a139959c-b6ba-42c0-8340-8fea54b733dc service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:45.834791 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a139959c-b6ba-42c0-8340-8fea54b733dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:45.847955 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a139959c-b6ba-42c0-8340-8fea54b733dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:45.849781 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a139959c-b6ba-42c0-8340-8fea54b733dc service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:45.866386 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 872/1777] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:25:45 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:25:46.166250 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd3faf6-b87f-4a76-b833-7465afd0c4fb None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:46.178486 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd3faf6-b87f-4a76-b833-7465afd0c4fb None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:46.183120 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dde0aa-033f-4aa6-a4e0-4e92d8c5e0e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e31f2728056b460fa474478eb85ca1d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:46.183921 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dde0aa-033f-4aa6-a4e0-4e92d8c5e0e6 service cinder] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:46.184271 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dde0aa-033f-4aa6-a4e0-4e92d8c5e0e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:46.184612 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dde0aa-033f-4aa6-a4e0-4e92d8c5e0e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:46.291812 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dde0aa-033f-4aa6-a4e0-4e92d8c5e0e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:46.294513 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dde0aa-033f-4aa6-a4e0-4e92d8c5e0e6 service cinder] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:46.320245 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 906/1778] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:25:46 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:25:46.379801 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a139959c-b6ba-42c0-8340-8fea54b733dc service cinder] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:46.403695 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:46.446815 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a139959c-b6ba-42c0-8340-8fea54b733dc service cinder] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:46.451663 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7bda1e-29bb-4b0b-a16e-7ebf1dac36c7 tempest-VolumeQuotaClassesTest-66973684 tempest-VolumeQuotaClassesTest-66973684-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a41964f0d1dd43a8b7b88dd8f68c7b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06cd80fe6def42e49240cb05b4ff853f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:46.452295 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7bda1e-29bb-4b0b-a16e-7ebf1dac36c7 tempest-VolumeQuotaClassesTest-66973684 tempest-VolumeQuotaClassesTest-66973684-project-admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:46.452582 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7bda1e-29bb-4b0b-a16e-7ebf1dac36c7 tempest-VolumeQuotaClassesTest-66973684 tempest-VolumeQuotaClassesTest-66973684-project-admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:46.452868 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7bda1e-29bb-4b0b-a16e-7ebf1dac36c7 tempest-VolumeQuotaClassesTest-66973684 tempest-VolumeQuotaClassesTest-66973684-project-admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:46.453576 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7bda1e-29bb-4b0b-a16e-7ebf1dac36c7 tempest-VolumeQuotaClassesTest-66973684 tempest-VolumeQuotaClassesTest-66973684-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:25:46.453888 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7bda1e-29bb-4b0b-a16e-7ebf1dac36c7 tempest-VolumeQuotaClassesTest-66973684 tempest-VolumeQuotaClassesTest-66973684-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:46.456312 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7bda1e-29bb-4b0b-a16e-7ebf1dac36c7 tempest-VolumeQuotaClassesTest-66973684 tempest-VolumeQuotaClassesTest-66973684-project-admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:46.470215 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 873/1779] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Aug 4 21:25:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:25:46.474746 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dde0aa-033f-4aa6-a4e0-4e92d8c5e0e6 service cinder] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:46.485365 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dde0aa-033f-4aa6-a4e0-4e92d8c5e0e6 service cinder] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:46.493438 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5b6133-150a-46bb-85e7-49a6619bfd65 tempest-VolumeQuotaClassesTest-66973684 tempest-VolumeQuotaClassesTest-66973684-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a41964f0d1dd43a8b7b88dd8f68c7b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06cd80fe6def42e49240cb05b4ff853f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:46.494377 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5b6133-150a-46bb-85e7-49a6619bfd65 tempest-VolumeQuotaClassesTest-66973684 tempest-VolumeQuotaClassesTest-66973684-project-admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:46.494519 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5b6133-150a-46bb-85e7-49a6619bfd65 tempest-VolumeQuotaClassesTest-66973684 tempest-VolumeQuotaClassesTest-66973684-project-admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:46.494853 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5b6133-150a-46bb-85e7-49a6619bfd65 tempest-VolumeQuotaClassesTest-66973684 tempest-VolumeQuotaClassesTest-66973684-project-admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:46.495963 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5b6133-150a-46bb-85e7-49a6619bfd65 tempest-VolumeQuotaClassesTest-66973684 tempest-VolumeQuotaClassesTest-66973684-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:46.500506 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5b6133-150a-46bb-85e7-49a6619bfd65 tempest-VolumeQuotaClassesTest-66973684 tempest-VolumeQuotaClassesTest-66973684-project-admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:46.588567 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 907/1780] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:25:46 2022] POST /identity/v3/projects => generated 355 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:25:46.594642 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7bda1e-29bb-4b0b-a16e-7ebf1dac36c7 tempest-VolumeQuotaClassesTest-66973684 tempest-VolumeQuotaClassesTest-66973684-project-admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:46.620356 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7bda1e-29bb-4b0b-a16e-7ebf1dac36c7 tempest-VolumeQuotaClassesTest-66973684 tempest-VolumeQuotaClassesTest-66973684-project-admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:46.730886 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:46.748124 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ddab9b-eacd-4e86-a037-5849693d2a5c tempest-VolumeQuotaClassesTest-66973684 tempest-VolumeQuotaClassesTest-66973684-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a41964f0d1dd43a8b7b88dd8f68c7b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06cd80fe6def42e49240cb05b4ff853f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:46.751426 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ddab9b-eacd-4e86-a037-5849693d2a5c tempest-VolumeQuotaClassesTest-66973684 tempest-VolumeQuotaClassesTest-66973684-project-admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:46.752104 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ddab9b-eacd-4e86-a037-5849693d2a5c tempest-VolumeQuotaClassesTest-66973684 tempest-VolumeQuotaClassesTest-66973684-project-admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:46.752700 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ddab9b-eacd-4e86-a037-5849693d2a5c tempest-VolumeQuotaClassesTest-66973684 tempest-VolumeQuotaClassesTest-66973684-project-admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:46.753696 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ddab9b-eacd-4e86-a037-5849693d2a5c tempest-VolumeQuotaClassesTest-66973684 tempest-VolumeQuotaClassesTest-66973684-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:25:46.754591 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ddab9b-eacd-4e86-a037-5849693d2a5c tempest-VolumeQuotaClassesTest-66973684 tempest-VolumeQuotaClassesTest-66973684-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:46.756686 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ddab9b-eacd-4e86-a037-5849693d2a5c tempest-VolumeQuotaClassesTest-66973684 tempest-VolumeQuotaClassesTest-66973684-project-admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:46.772706 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 874/1781] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Aug 4 21:25:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:25:46.865892 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5b6133-150a-46bb-85e7-49a6619bfd65 tempest-VolumeQuotaClassesTest-66973684 tempest-VolumeQuotaClassesTest-66973684-project-admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:46.879230 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5b6133-150a-46bb-85e7-49a6619bfd65 tempest-VolumeQuotaClassesTest-66973684 tempest-VolumeQuotaClassesTest-66973684-project-admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:46.886864 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdf2c53-48cc-4543-b1d6-23d57a6a821e tempest-VolumeQuotaClassesTest-66973684 tempest-VolumeQuotaClassesTest-66973684-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a41964f0d1dd43a8b7b88dd8f68c7b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06cd80fe6def42e49240cb05b4ff853f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:46.887534 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdf2c53-48cc-4543-b1d6-23d57a6a821e tempest-VolumeQuotaClassesTest-66973684 tempest-VolumeQuotaClassesTest-66973684-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:46.887826 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdf2c53-48cc-4543-b1d6-23d57a6a821e tempest-VolumeQuotaClassesTest-66973684 tempest-VolumeQuotaClassesTest-66973684-project-admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:46.891276 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdf2c53-48cc-4543-b1d6-23d57a6a821e tempest-VolumeQuotaClassesTest-66973684 tempest-VolumeQuotaClassesTest-66973684-project-admin] PATH_INFO: `/v3/projects/e0990c31e4eb4a72abdad0e40dce2ae8` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:46.891276 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdf2c53-48cc-4543-b1d6-23d57a6a821e tempest-VolumeQuotaClassesTest-66973684 tempest-VolumeQuotaClassesTest-66973684-project-admin] RBAC: Authorizing `identity:delete_project(project_id=e0990c31e4eb4a72abdad0e40dce2ae8)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:46.905066 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdf2c53-48cc-4543-b1d6-23d57a6a821e tempest-VolumeQuotaClassesTest-66973684 tempest-VolumeQuotaClassesTest-66973684-project-admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:47.194266 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-6fdf2c53-48cc-4543-b1d6-23d57a6a821e tempest-VolumeQuotaClassesTest-66973684 tempest-VolumeQuotaClassesTest-66973684-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0990c31e4eb4a72abdad0e40dce2ae8', 'request_id': 'req-6fdf2c53-48cc-4543-b1d6-23d57a6a821e'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:47.203542 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-6fdf2c53-48cc-4543-b1d6-23d57a6a821e tempest-VolumeQuotaClassesTest-66973684 tempest-VolumeQuotaClassesTest-66973684-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0990c31e4eb4a72abdad0e40dce2ae8', 'request_id': 'req-6fdf2c53-48cc-4543-b1d6-23d57a6a821e'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:47.324960 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-6fdf2c53-48cc-4543-b1d6-23d57a6a821e tempest-VolumeQuotaClassesTest-66973684 tempest-VolumeQuotaClassesTest-66973684-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0990c31e4eb4a72abdad0e40dce2ae8', 'request_id': 'req-6fdf2c53-48cc-4543-b1d6-23d57a6a821e'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:47.344138 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-6fdf2c53-48cc-4543-b1d6-23d57a6a821e tempest-VolumeQuotaClassesTest-66973684 tempest-VolumeQuotaClassesTest-66973684-project-admin] The token cache is being invalidate because project e0990c31e4eb4a72abdad0e40dce2ae8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:25:47.344561 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-6fdf2c53-48cc-4543-b1d6-23d57a6a821e tempest-VolumeQuotaClassesTest-66973684 tempest-VolumeQuotaClassesTest-66973684-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:47.346318 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-6fdf2c53-48cc-4543-b1d6-23d57a6a821e tempest-VolumeQuotaClassesTest-66973684 tempest-VolumeQuotaClassesTest-66973684-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:47.351716 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 908/1782] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:25:46 2022] DELETE /identity/v3/projects/e0990c31e4eb4a72abdad0e40dce2ae8 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:25:47.518893 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:48.355149 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ddab9b-eacd-4e86-a037-5849693d2a5c tempest-VolumeQuotaClassesTest-66973684 tempest-VolumeQuotaClassesTest-66973684-project-admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:48.357812 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-13ddab9b-eacd-4e86-a037-5849693d2a5c tempest-VolumeQuotaClassesTest-66973684 tempest-VolumeQuotaClassesTest-66973684-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:48.481481 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ddab9b-eacd-4e86-a037-5849693d2a5c tempest-VolumeQuotaClassesTest-66973684 tempest-VolumeQuotaClassesTest-66973684-project-admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:48.620554 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca68e87-2edb-425a-a5a4-7cd01b2f7b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:48.621359 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca68e87-2edb-425a-a5a4-7cd01b2f7b59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:48.621644 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca68e87-2edb-425a-a5a4-7cd01b2f7b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:48.621924 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca68e87-2edb-425a-a5a4-7cd01b2f7b59 admin admin] PATH_INFO: `/v3/users/837f72c758964b2eb11786cd115baf5f` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:48.623449 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca68e87-2edb-425a-a5a4-7cd01b2f7b59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=837f72c758964b2eb11786cd115baf5f)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:48.627599 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca68e87-2edb-425a-a5a4-7cd01b2f7b59 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:48.974241 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-fca68e87-2edb-425a-a5a4-7cd01b2f7b59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '837f72c758964b2eb11786cd115baf5f'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:49.026400 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:49.027431 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-fca68e87-2edb-425a-a5a4-7cd01b2f7b59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '837f72c758964b2eb11786cd115baf5f'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:49.033353 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-fca68e87-2edb-425a-a5a4-7cd01b2f7b59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '837f72c758964b2eb11786cd115baf5f'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:49.036553 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-fca68e87-2edb-425a-a5a4-7cd01b2f7b59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '837f72c758964b2eb11786cd115baf5f'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:49.037637 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-fca68e87-2edb-425a-a5a4-7cd01b2f7b59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '837f72c758964b2eb11786cd115baf5f'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:49.088395 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdf2c53-48cc-4543-b1d6-23d57a6a821e tempest-VolumeQuotaClassesTest-66973684 tempest-VolumeQuotaClassesTest-66973684-project-admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:49.090602 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-6fdf2c53-48cc-4543-b1d6-23d57a6a821e tempest-VolumeQuotaClassesTest-66973684 tempest-VolumeQuotaClassesTest-66973684-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:49.124180 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 875/1783] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:25:48 2022] DELETE /identity/v3/users/837f72c758964b2eb11786cd115baf5f => generated 0 bytes in 769 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:25:49.127932 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ea3c5-6e82-46c8-ad78-0d50b741b2e3 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:49.128358 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ea3c5-6e82-46c8-ad78-0d50b741b2e3 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:49.128743 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ea3c5-6e82-46c8-ad78-0d50b741b2e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:49.132149 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdf2c53-48cc-4543-b1d6-23d57a6a821e tempest-VolumeQuotaClassesTest-66973684 tempest-VolumeQuotaClassesTest-66973684-project-admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:49.172676 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-0f7ea3c5-6e82-46c8-ad78-0d50b741b2e3 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:25:49.177103 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-0f7ea3c5-6e82-46c8-ad78-0d50b741b2e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:49.321197 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26e5629-5f5c-44cb-b455-5bd6fa5a1f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:49.321884 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e5629-5f5c-44cb-b455-5bd6fa5a1f39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:49.322213 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e5629-5f5c-44cb-b455-5bd6fa5a1f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:49.322664 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e5629-5f5c-44cb-b455-5bd6fa5a1f39 admin admin] PATH_INFO: `/v3/users/ddeb36626a5f4a0aa4a6ae73d5a30642` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:49.325486 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:49.327930 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26e5629-5f5c-44cb-b455-5bd6fa5a1f39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ddeb36626a5f4a0aa4a6ae73d5a30642)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:49.329998 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26e5629-5f5c-44cb-b455-5bd6fa5a1f39 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:49.342898 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 876/1784] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:25:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:25:49.380582 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7ea3c5-6e82-46c8-ad78-0d50b741b2e3 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:49.382062 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-0f7ea3c5-6e82-46c8-ad78-0d50b741b2e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:49.492977 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7ea3c5-6e82-46c8-ad78-0d50b741b2e3 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:49.497464 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7586bc66-db92-4b85-a829-b2d6325a3bfc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:49.498949 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7586bc66-db92-4b85-a829-b2d6325a3bfc service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:49.502709 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7586bc66-db92-4b85-a829-b2d6325a3bfc service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:49.502709 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7586bc66-db92-4b85-a829-b2d6325a3bfc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:49.512954 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7586bc66-db92-4b85-a829-b2d6325a3bfc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:49.527824 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7586bc66-db92-4b85-a829-b2d6325a3bfc service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:49.557640 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 877/1785] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:25:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:25:49.619554 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-f26e5629-5f5c-44cb-b455-5bd6fa5a1f39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ddeb36626a5f4a0aa4a6ae73d5a30642'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:49.661927 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-f26e5629-5f5c-44cb-b455-5bd6fa5a1f39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ddeb36626a5f4a0aa4a6ae73d5a30642'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:49.663716 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-f26e5629-5f5c-44cb-b455-5bd6fa5a1f39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ddeb36626a5f4a0aa4a6ae73d5a30642'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:49.666726 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-f26e5629-5f5c-44cb-b455-5bd6fa5a1f39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ddeb36626a5f4a0aa4a6ae73d5a30642'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:49.760485 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-f26e5629-5f5c-44cb-b455-5bd6fa5a1f39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ddeb36626a5f4a0aa4a6ae73d5a30642'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:49.809908 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 909/1786] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:25:49 2022] DELETE /identity/v3/users/ddeb36626a5f4a0aa4a6ae73d5a30642 => generated 0 bytes in 719 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:25:49.811736 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a3f830-7d3e-4bcc-94f4-fb69b46e4383 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:49.812029 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a3f830-7d3e-4bcc-94f4-fb69b46e4383 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:49.812308 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a3f830-7d3e-4bcc-94f4-fb69b46e4383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:49.855068 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-14a3f830-7d3e-4bcc-94f4-fb69b46e4383 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:25:49.856612 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-14a3f830-7d3e-4bcc-94f4-fb69b46e4383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:49.934779 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 878/1787] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:25:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:25:49.947849 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26e5629-5f5c-44cb-b455-5bd6fa5a1f39 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:49.952447 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-f26e5629-5f5c-44cb-b455-5bd6fa5a1f39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:50.037390 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26e5629-5f5c-44cb-b455-5bd6fa5a1f39 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:50.052721 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7fcf80-9453-4359-abdf-2198b22b6720 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:50.053700 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7fcf80-9453-4359-abdf-2198b22b6720 service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:50.053990 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7fcf80-9453-4359-abdf-2198b22b6720 service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:50.054266 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7fcf80-9453-4359-abdf-2198b22b6720 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:50.082873 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7fcf80-9453-4359-abdf-2198b22b6720 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:50.085001 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7fcf80-9453-4359-abdf-2198b22b6720 service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:50.116842 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 910/1788] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:25:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:25:50.593162 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e38ffbc-98ff-4381-b0cd-63ab9e121596 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:50.593696 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e38ffbc-98ff-4381-b0cd-63ab9e121596 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:50.593696 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e38ffbc-98ff-4381-b0cd-63ab9e121596 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:50.597978 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 879/1789] 10.222.0.25 () {54 vars in 921 bytes} [Thu Aug 4 21:25:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:25:50.613010 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a54ca08-ee91-4ca3-abc4-e9b4a5026072 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:50.613797 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a54ca08-ee91-4ca3-abc4-e9b4a5026072 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:50.614393 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a54ca08-ee91-4ca3-abc4-e9b4a5026072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:50.664648 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-0a54ca08-ee91-4ca3-abc4-e9b4a5026072 None None] MFA Rules not processed for user `6a2ce1df87fa47e5be301916bf3245fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:25:50.666892 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-0a54ca08-ee91-4ca3-abc4-e9b4a5026072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:50.779606 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 911/1790] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Aug 4 21:25:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:25:51.006811 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:51.077168 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e38ffbc-98ff-4381-b0cd-63ab9e121596 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:51.079503 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-6e38ffbc-98ff-4381-b0cd-63ab9e121596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:51.109301 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e38ffbc-98ff-4381-b0cd-63ab9e121596 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:51.119554 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc7edda-4c08-4d75-8554-7cd845844924 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a2ce1df87fa47e5be301916bf3245fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:51.120588 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc7edda-4c08-4d75-8554-7cd845844924 service nova] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:51.121244 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc7edda-4c08-4d75-8554-7cd845844924 service nova] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:51.121709 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc7edda-4c08-4d75-8554-7cd845844924 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:51.137627 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc7edda-4c08-4d75-8554-7cd845844924 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:51.140128 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc7edda-4c08-4d75-8554-7cd845844924 service nova] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:51.171780 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 880/1791] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:25:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:25:51.401314 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a54ca08-ee91-4ca3-abc4-e9b4a5026072 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:51.406035 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-0a54ca08-ee91-4ca3-abc4-e9b4a5026072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:51.430483 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a54ca08-ee91-4ca3-abc4-e9b4a5026072 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:51.439394 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6c0ee8-c02a-41e1-adcf-ee8759063c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:51.440269 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6c0ee8-c02a-41e1-adcf-ee8759063c1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:51.440648 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6c0ee8-c02a-41e1-adcf-ee8759063c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:51.441343 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6c0ee8-c02a-41e1-adcf-ee8759063c1c admin admin] PATH_INFO: `/v3/projects/ea1fd140c0414501bb7765130c4c0651` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:51.445943 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6c0ee8-c02a-41e1-adcf-ee8759063c1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea1fd140c0414501bb7765130c4c0651)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:51.449377 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6c0ee8-c02a-41e1-adcf-ee8759063c1c admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:51.595302 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-3e6c0ee8-c02a-41e1-adcf-ee8759063c1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea1fd140c0414501bb7765130c4c0651', 'request_id': 'req-3e6c0ee8-c02a-41e1-adcf-ee8759063c1c'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:51.601376 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-3e6c0ee8-c02a-41e1-adcf-ee8759063c1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea1fd140c0414501bb7765130c4c0651', 'request_id': 'req-3e6c0ee8-c02a-41e1-adcf-ee8759063c1c'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:51.683771 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-3e6c0ee8-c02a-41e1-adcf-ee8759063c1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea1fd140c0414501bb7765130c4c0651', 'request_id': 'req-3e6c0ee8-c02a-41e1-adcf-ee8759063c1c'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:51.704674 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-3e6c0ee8-c02a-41e1-adcf-ee8759063c1c admin admin] The token cache is being invalidate because project ea1fd140c0414501bb7765130c4c0651 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:25:51.705488 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-3e6c0ee8-c02a-41e1-adcf-ee8759063c1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:51.706236 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-3e6c0ee8-c02a-41e1-adcf-ee8759063c1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:51.708026 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 912/1792] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:25:51 2022] DELETE /identity/v3/projects/ea1fd140c0414501bb7765130c4c0651 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:25:51.711986 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc7edda-4c08-4d75-8554-7cd845844924 service nova] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:51.724129 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc7edda-4c08-4d75-8554-7cd845844924 service nova] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:51.820566 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57e9ef0-e470-41d5-b9bb-99ba094e81b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:51.821841 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57e9ef0-e470-41d5-b9bb-99ba094e81b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:51.822487 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57e9ef0-e470-41d5-b9bb-99ba094e81b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:51.823079 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57e9ef0-e470-41d5-b9bb-99ba094e81b1 admin admin] PATH_INFO: `/v3/users/a41964f0d1dd43a8b7b88dd8f68c7b24` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:51.825972 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57e9ef0-e470-41d5-b9bb-99ba094e81b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a41964f0d1dd43a8b7b88dd8f68c7b24)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:51.829665 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57e9ef0-e470-41d5-b9bb-99ba094e81b1 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:52.094233 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-d57e9ef0-e470-41d5-b9bb-99ba094e81b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a41964f0d1dd43a8b7b88dd8f68c7b24'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:52.139247 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-d57e9ef0-e470-41d5-b9bb-99ba094e81b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a41964f0d1dd43a8b7b88dd8f68c7b24'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:52.142247 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-d57e9ef0-e470-41d5-b9bb-99ba094e81b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a41964f0d1dd43a8b7b88dd8f68c7b24'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:52.170129 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-d57e9ef0-e470-41d5-b9bb-99ba094e81b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a41964f0d1dd43a8b7b88dd8f68c7b24'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:52.172413 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-d57e9ef0-e470-41d5-b9bb-99ba094e81b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a41964f0d1dd43a8b7b88dd8f68c7b24'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:52.293595 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 881/1793] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:25:51 2022] DELETE /identity/v3/users/a41964f0d1dd43a8b7b88dd8f68c7b24 => generated 0 bytes in 579 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:25:52.313371 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6c0ee8-c02a-41e1-adcf-ee8759063c1c admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:52.319520 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-3e6c0ee8-c02a-41e1-adcf-ee8759063c1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:52.353738 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6c0ee8-c02a-41e1-adcf-ee8759063c1c admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:52.451550 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6d2706-738f-4045-b6e6-c4fbdb06280d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:52.452224 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6d2706-738f-4045-b6e6-c4fbdb06280d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:52.452494 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6d2706-738f-4045-b6e6-c4fbdb06280d admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:52.453108 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6d2706-738f-4045-b6e6-c4fbdb06280d admin admin] PATH_INFO: `/v3/projects/1d922c64da53449089653c1b53e09216` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:52.454866 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6d2706-738f-4045-b6e6-c4fbdb06280d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d922c64da53449089653c1b53e09216)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:52.457503 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6d2706-738f-4045-b6e6-c4fbdb06280d admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:52.628255 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-de6d2706-738f-4045-b6e6-c4fbdb06280d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d922c64da53449089653c1b53e09216', 'request_id': 'req-de6d2706-738f-4045-b6e6-c4fbdb06280d'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:52.641258 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-de6d2706-738f-4045-b6e6-c4fbdb06280d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d922c64da53449089653c1b53e09216', 'request_id': 'req-de6d2706-738f-4045-b6e6-c4fbdb06280d'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:52.770125 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-de6d2706-738f-4045-b6e6-c4fbdb06280d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d922c64da53449089653c1b53e09216', 'request_id': 'req-de6d2706-738f-4045-b6e6-c4fbdb06280d'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:52.785627 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-de6d2706-738f-4045-b6e6-c4fbdb06280d admin admin] The token cache is being invalidate because project 1d922c64da53449089653c1b53e09216 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:25:52.785932 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-de6d2706-738f-4045-b6e6-c4fbdb06280d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:52.786609 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-de6d2706-738f-4045-b6e6-c4fbdb06280d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:52.788218 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 913/1794] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:25:52 2022] DELETE /identity/v3/projects/1d922c64da53449089653c1b53e09216 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:25:52.792234 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57e9ef0-e470-41d5-b9bb-99ba094e81b1 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:52.793692 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-d57e9ef0-e470-41d5-b9bb-99ba094e81b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:52.807040 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57e9ef0-e470-41d5-b9bb-99ba094e81b1 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:52.880011 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059f8bfd-c01d-4306-8c45-1e8a50ca9210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:52.880800 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059f8bfd-c01d-4306-8c45-1e8a50ca9210 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:52.881200 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059f8bfd-c01d-4306-8c45-1e8a50ca9210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:52.881494 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059f8bfd-c01d-4306-8c45-1e8a50ca9210 admin admin] PATH_INFO: `/v3/users/ff862c510c3f4014bcd4c4ee41444f1b` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:52.882943 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059f8bfd-c01d-4306-8c45-1e8a50ca9210 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff862c510c3f4014bcd4c4ee41444f1b)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:52.885353 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059f8bfd-c01d-4306-8c45-1e8a50ca9210 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:52.910820 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:53.024010 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6d2706-738f-4045-b6e6-c4fbdb06280d admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:53.025575 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-de6d2706-738f-4045-b6e6-c4fbdb06280d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:53.044031 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6d2706-738f-4045-b6e6-c4fbdb06280d admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:53.058535 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac52143-cfb7-4e88-bd7e-4dc9e3268f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:53.059077 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac52143-cfb7-4e88-bd7e-4dc9e3268f48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:53.059401 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac52143-cfb7-4e88-bd7e-4dc9e3268f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:53.059739 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac52143-cfb7-4e88-bd7e-4dc9e3268f48 admin admin] PATH_INFO: `/v3/projects/06cd80fe6def42e49240cb05b4ff853f` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:53.060980 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac52143-cfb7-4e88-bd7e-4dc9e3268f48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06cd80fe6def42e49240cb05b4ff853f)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:53.064103 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac52143-cfb7-4e88-bd7e-4dc9e3268f48 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:53.125613 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-059f8bfd-c01d-4306-8c45-1e8a50ca9210 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff862c510c3f4014bcd4c4ee41444f1b'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:53.143225 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-059f8bfd-c01d-4306-8c45-1e8a50ca9210 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff862c510c3f4014bcd4c4ee41444f1b'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:53.144099 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-059f8bfd-c01d-4306-8c45-1e8a50ca9210 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff862c510c3f4014bcd4c4ee41444f1b'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:53.151769 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-059f8bfd-c01d-4306-8c45-1e8a50ca9210 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff862c510c3f4014bcd4c4ee41444f1b'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:53.152639 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-059f8bfd-c01d-4306-8c45-1e8a50ca9210 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff862c510c3f4014bcd4c4ee41444f1b'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:53.183233 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-eac52143-cfb7-4e88-bd7e-4dc9e3268f48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06cd80fe6def42e49240cb05b4ff853f', 'request_id': 'req-eac52143-cfb7-4e88-bd7e-4dc9e3268f48'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:53.190768 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-eac52143-cfb7-4e88-bd7e-4dc9e3268f48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06cd80fe6def42e49240cb05b4ff853f', 'request_id': 'req-eac52143-cfb7-4e88-bd7e-4dc9e3268f48'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:53.239274 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 882/1795] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:25:52 2022] DELETE /identity/v3/users/ff862c510c3f4014bcd4c4ee41444f1b => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:25:53.300461 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-eac52143-cfb7-4e88-bd7e-4dc9e3268f48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06cd80fe6def42e49240cb05b4ff853f', 'request_id': 'req-eac52143-cfb7-4e88-bd7e-4dc9e3268f48'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:53.321905 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-eac52143-cfb7-4e88-bd7e-4dc9e3268f48 admin admin] The token cache is being invalidate because project 06cd80fe6def42e49240cb05b4ff853f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:25:53.322408 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-eac52143-cfb7-4e88-bd7e-4dc9e3268f48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:53.323694 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-eac52143-cfb7-4e88-bd7e-4dc9e3268f48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:53.325390 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 914/1796] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:25:53 2022] DELETE /identity/v3/projects/06cd80fe6def42e49240cb05b4ff853f => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:25:53.331589 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838404f4-89ae-49eb-85ea-b28367102ced None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:53.331589 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838404f4-89ae-49eb-85ea-b28367102ced None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:53.337082 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838404f4-89ae-49eb-85ea-b28367102ced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:53.352710 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-838404f4-89ae-49eb-85ea-b28367102ced None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:25:53.353963 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-838404f4-89ae-49eb-85ea-b28367102ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:53.428973 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 883/1797] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:25:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:25:53.440481 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc8a03e-7aee-4eb0-9512-ee11b9d65fa1 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:53.441251 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc8a03e-7aee-4eb0-9512-ee11b9d65fa1 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:53.441806 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc8a03e-7aee-4eb0-9512-ee11b9d65fa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:53.463715 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-0bc8a03e-7aee-4eb0-9512-ee11b9d65fa1 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:25:53.464952 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-0bc8a03e-7aee-4eb0-9512-ee11b9d65fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:53.473557 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 915/1798] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:25:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:25:53.478078 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838404f4-89ae-49eb-85ea-b28367102ced None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:53.504010 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838404f4-89ae-49eb-85ea-b28367102ced None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:53.510385 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183721f4-1646-4ae4-9409-2c2aa2c82c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:53.511517 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183721f4-1646-4ae4-9409-2c2aa2c82c1c admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:53.511782 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183721f4-1646-4ae4-9409-2c2aa2c82c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:53.512150 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183721f4-1646-4ae4-9409-2c2aa2c82c1c admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:53.514163 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183721f4-1646-4ae4-9409-2c2aa2c82c1c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:25:53.514478 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183721f4-1646-4ae4-9409-2c2aa2c82c1c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:53.517052 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183721f4-1646-4ae4-9409-2c2aa2c82c1c admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:53.536962 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 884/1799] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Aug 4 21:25:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:25:53.541365 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc8a03e-7aee-4eb0-9512-ee11b9d65fa1 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:53.552440 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc8a03e-7aee-4eb0-9512-ee11b9d65fa1 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:53.562774 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a94e9e3-ea20-471a-96d3-5e7e856406ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:53.563745 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a94e9e3-ea20-471a-96d3-5e7e856406ec admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:53.564029 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a94e9e3-ea20-471a-96d3-5e7e856406ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:53.564300 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a94e9e3-ea20-471a-96d3-5e7e856406ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:53.565044 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a94e9e3-ea20-471a-96d3-5e7e856406ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:53.566994 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a94e9e3-ea20-471a-96d3-5e7e856406ec admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:53.680262 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 916/1800] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:25:53 2022] POST /identity/v3/projects => generated 371 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:25:53.684875 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183721f4-1646-4ae4-9409-2c2aa2c82c1c admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:53.696231 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183721f4-1646-4ae4-9409-2c2aa2c82c1c admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:53.773981 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecf809d-c3fa-40cd-a89d-e88c5394b1b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:53.773981 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf809d-c3fa-40cd-a89d-e88c5394b1b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:53.773981 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf809d-c3fa-40cd-a89d-e88c5394b1b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:53.773981 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf809d-c3fa-40cd-a89d-e88c5394b1b4 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:53.774439 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecf809d-c3fa-40cd-a89d-e88c5394b1b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:53.774439 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecf809d-c3fa-40cd-a89d-e88c5394b1b4 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:53.801312 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a94e9e3-ea20-471a-96d3-5e7e856406ec admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:53.803169 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-3a94e9e3-ea20-471a-96d3-5e7e856406ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:53.821214 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a94e9e3-ea20-471a-96d3-5e7e856406ec admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:53.826684 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d758dd-1d50-4bd4-9f16-176ed3510a07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:53.827383 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d758dd-1d50-4bd4-9f16-176ed3510a07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:53.827644 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d758dd-1d50-4bd4-9f16-176ed3510a07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:53.828128 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d758dd-1d50-4bd4-9f16-176ed3510a07 admin admin] PATH_INFO: `/v3/projects/83f31a7bb57841b89048f9b89e17dee2` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:53.830067 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d758dd-1d50-4bd4-9f16-176ed3510a07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83f31a7bb57841b89048f9b89e17dee2)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:53.832124 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d758dd-1d50-4bd4-9f16-176ed3510a07 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:53.851632 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 885/1801] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:25:53 2022] POST /identity/v3/users => generated 286 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:25:53.851632 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecf809d-c3fa-40cd-a89d-e88c5394b1b4 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:53.865038 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecf809d-c3fa-40cd-a89d-e88c5394b1b4 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:53.870317 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8463f3ed-de9f-4998-bad8-78e544945167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:53.870966 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8463f3ed-de9f-4998-bad8-78e544945167 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:53.871252 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8463f3ed-de9f-4998-bad8-78e544945167 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:53.871686 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8463f3ed-de9f-4998-bad8-78e544945167 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:53.872338 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8463f3ed-de9f-4998-bad8-78e544945167 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:53.873952 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8463f3ed-de9f-4998-bad8-78e544945167 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:53.886734 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 886/1802] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:25:53 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:25:53.895639 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8463f3ed-de9f-4998-bad8-78e544945167 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:53.906386 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8463f3ed-de9f-4998-bad8-78e544945167 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:53.911765 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f576b57-1257-44d3-957b-414cd6e46cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:53.912468 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f576b57-1257-44d3-957b-414cd6e46cbf admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:53.912751 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f576b57-1257-44d3-957b-414cd6e46cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:53.913073 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f576b57-1257-44d3-957b-414cd6e46cbf admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:53.913834 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f576b57-1257-44d3-957b-414cd6e46cbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:53.915798 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f576b57-1257-44d3-957b-414cd6e46cbf admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:53.927874 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 887/1803] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:25:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:25:53.933473 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f576b57-1257-44d3-957b-414cd6e46cbf admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:53.944739 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f576b57-1257-44d3-957b-414cd6e46cbf admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:53.955169 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af15a8d5-cdf4-4db8-932f-2c0650628ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:53.955169 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af15a8d5-cdf4-4db8-932f-2c0650628ba9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:53.955169 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af15a8d5-cdf4-4db8-932f-2c0650628ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:53.955169 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af15a8d5-cdf4-4db8-932f-2c0650628ba9 admin admin] PATH_INFO: `/v3/projects/c6d2209282e9415688759b7067c420e3/users/7c67b93ddfef492a9c834a4f6dc6308b/roles/8feebc60363c4feb870718b1721e5324` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:53.974941 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af15a8d5-cdf4-4db8-932f-2c0650628ba9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6d2209282e9415688759b7067c420e3, user_id=7c67b93ddfef492a9c834a4f6dc6308b, role_id=8feebc60363c4feb870718b1721e5324)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:53.978694 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af15a8d5-cdf4-4db8-932f-2c0650628ba9 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:53.979291 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-34d758dd-1d50-4bd4-9f16-176ed3510a07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83f31a7bb57841b89048f9b89e17dee2', 'request_id': 'req-34d758dd-1d50-4bd4-9f16-176ed3510a07'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:53.987341 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-34d758dd-1d50-4bd4-9f16-176ed3510a07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83f31a7bb57841b89048f9b89e17dee2', 'request_id': 'req-34d758dd-1d50-4bd4-9f16-176ed3510a07'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:54.050132 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-34d758dd-1d50-4bd4-9f16-176ed3510a07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83f31a7bb57841b89048f9b89e17dee2', 'request_id': 'req-34d758dd-1d50-4bd4-9f16-176ed3510a07'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:54.057170 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 888/1804] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:25:53 2022] PUT /identity/v3/projects/c6d2209282e9415688759b7067c420e3/users/7c67b93ddfef492a9c834a4f6dc6308b/roles/8feebc60363c4feb870718b1721e5324 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:25:54.064143 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-34d758dd-1d50-4bd4-9f16-176ed3510a07 admin admin] The token cache is being invalidate because project 83f31a7bb57841b89048f9b89e17dee2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:25:54.064143 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-34d758dd-1d50-4bd4-9f16-176ed3510a07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:54.064623 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-34d758dd-1d50-4bd4-9f16-176ed3510a07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:25:54.066436 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 917/1805] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:25:53 2022] DELETE /identity/v3/projects/83f31a7bb57841b89048f9b89e17dee2 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:25:54.069208 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b81ef8-f986-44fb-a1af-5406fbf22004 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:54.069631 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b81ef8-f986-44fb-a1af-5406fbf22004 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:54.070171 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b81ef8-f986-44fb-a1af-5406fbf22004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:54.071083 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16c638d-de33-4bfd-81aa-28fa30bf4729 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:54.071404 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16c638d-de33-4bfd-81aa-28fa30bf4729 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:54.071689 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16c638d-de33-4bfd-81aa-28fa30bf4729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:54.092711 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-f16c638d-de33-4bfd-81aa-28fa30bf4729 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:25:54.096585 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-f16c638d-de33-4bfd-81aa-28fa30bf4729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:54.101375 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-e0b81ef8-f986-44fb-a1af-5406fbf22004 None None] MFA Rules not processed for user `7c67b93ddfef492a9c834a4f6dc6308b`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:25:54.102551 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-e0b81ef8-f986-44fb-a1af-5406fbf22004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:54.156422 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 918/1806] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:25:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:25:54.164956 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca22705-c19b-48b6-8bcc-ef22a89bcceb None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:54.165463 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca22705-c19b-48b6-8bcc-ef22a89bcceb None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:54.167401 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca22705-c19b-48b6-8bcc-ef22a89bcceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:54.174792 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 889/1807] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:25:54 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:25:54.190521 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-eca22705-c19b-48b6-8bcc-ef22a89bcceb None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:25:54.191975 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-eca22705-c19b-48b6-8bcc-ef22a89bcceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:54.192561 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b81ef8-f986-44fb-a1af-5406fbf22004 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:54.194151 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-e0b81ef8-f986-44fb-a1af-5406fbf22004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:54.201101 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 919/1808] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:25:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:25:54.205828 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca22705-c19b-48b6-8bcc-ef22a89bcceb None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:54.215021 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca22705-c19b-48b6-8bcc-ef22a89bcceb None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:54.228204 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948bb451-4d96-4ebd-b930-51e2319685b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:54.228809 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948bb451-4d96-4ebd-b930-51e2319685b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:54.229101 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948bb451-4d96-4ebd-b930-51e2319685b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:54.229383 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948bb451-4d96-4ebd-b930-51e2319685b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:54.230455 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948bb451-4d96-4ebd-b930-51e2319685b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:25:54.230455 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948bb451-4d96-4ebd-b930-51e2319685b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:54.234107 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948bb451-4d96-4ebd-b930-51e2319685b3 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:54.252255 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 920/1809] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Aug 4 21:25:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:25:54.257194 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948bb451-4d96-4ebd-b930-51e2319685b3 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:54.271814 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948bb451-4d96-4ebd-b930-51e2319685b3 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:54.278198 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bdfc50-f1bc-44d2-acfc-ba06ba8d2b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:54.278852 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bdfc50-f1bc-44d2-acfc-ba06ba8d2b8e admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:54.279153 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bdfc50-f1bc-44d2-acfc-ba06ba8d2b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:54.279443 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bdfc50-f1bc-44d2-acfc-ba06ba8d2b8e admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:54.281246 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bdfc50-f1bc-44d2-acfc-ba06ba8d2b8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:54.285234 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bdfc50-f1bc-44d2-acfc-ba06ba8d2b8e admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:54.303714 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b81ef8-f986-44fb-a1af-5406fbf22004 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:54.308713 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5de91c3-d920-4f5b-9ef7-15d8493b6b25 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6885c7d929f4f8e96d3890724c3b53f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:54.310249 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5de91c3-d920-4f5b-9ef7-15d8493b6b25 service glance] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:54.310667 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5de91c3-d920-4f5b-9ef7-15d8493b6b25 service glance] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:54.311050 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5de91c3-d920-4f5b-9ef7-15d8493b6b25 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:54.321754 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5de91c3-d920-4f5b-9ef7-15d8493b6b25 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:54.323443 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5de91c3-d920-4f5b-9ef7-15d8493b6b25 service glance] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:54.337785 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 890/1810] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:25:54 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:25:54.467583 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 921/1811] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:25:54 2022] POST /identity/v3/projects => generated 369 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:25:54.472667 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5de91c3-d920-4f5b-9ef7-15d8493b6b25 service glance] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:54.485082 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5de91c3-d920-4f5b-9ef7-15d8493b6b25 service glance] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:54.573773 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9c2be3-a8e8-4435-97ca-ac8615cef2e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:54.574592 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9c2be3-a8e8-4435-97ca-ac8615cef2e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:54.574727 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9c2be3-a8e8-4435-97ca-ac8615cef2e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:54.575053 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9c2be3-a8e8-4435-97ca-ac8615cef2e3 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:54.576403 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9c2be3-a8e8-4435-97ca-ac8615cef2e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:54.577801 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9c2be3-a8e8-4435-97ca-ac8615cef2e3 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:54.594042 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:DELETE=7,keystone:SELECT=88,keystone:INSERT=5 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:54.715031 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 891/1812] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:25:54 2022] POST /identity/v3/users => generated 285 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:25:54.719708 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bdfc50-f1bc-44d2-acfc-ba06ba8d2b8e admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:54.730154 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bdfc50-f1bc-44d2-acfc-ba06ba8d2b8e admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:54.735132 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bff737-714b-47d1-854f-81f42e490340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:54.735787 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bff737-714b-47d1-854f-81f42e490340 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:54.736068 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bff737-714b-47d1-854f-81f42e490340 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:54.736342 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bff737-714b-47d1-854f-81f42e490340 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:54.737104 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bff737-714b-47d1-854f-81f42e490340 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:54.738771 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bff737-714b-47d1-854f-81f42e490340 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:54.751355 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 922/1813] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:25:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:25:54.754083 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9c2be3-a8e8-4435-97ca-ac8615cef2e3 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:54.764426 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9c2be3-a8e8-4435-97ca-ac8615cef2e3 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:54.770198 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82705b68-4f68-4db9-a8c3-bad369da492c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:54.770772 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82705b68-4f68-4db9-a8c3-bad369da492c admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:54.771048 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82705b68-4f68-4db9-a8c3-bad369da492c admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:54.771354 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82705b68-4f68-4db9-a8c3-bad369da492c admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:54.772037 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82705b68-4f68-4db9-a8c3-bad369da492c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:54.773644 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82705b68-4f68-4db9-a8c3-bad369da492c admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:54.785671 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 892/1814] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:25:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:25:54.790963 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bff737-714b-47d1-854f-81f42e490340 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:54.808205 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bff737-714b-47d1-854f-81f42e490340 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:54.808205 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e59da2-3f0d-43d5-b8d8-10d48cf0d923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:54.809182 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e59da2-3f0d-43d5-b8d8-10d48cf0d923 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:54.809464 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e59da2-3f0d-43d5-b8d8-10d48cf0d923 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:54.809743 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e59da2-3f0d-43d5-b8d8-10d48cf0d923 admin admin] PATH_INFO: `/v3/projects/d011c2789975455ea85b49909d4231cb/users/ea3dad37f03b4dfe9f1019f391914cbc/roles/8feebc60363c4feb870718b1721e5324` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:54.826232 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e59da2-3f0d-43d5-b8d8-10d48cf0d923 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d011c2789975455ea85b49909d4231cb, user_id=ea3dad37f03b4dfe9f1019f391914cbc, role_id=8feebc60363c4feb870718b1721e5324)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:54.829022 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e59da2-3f0d-43d5-b8d8-10d48cf0d923 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:54.925317 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 923/1815] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:25:54 2022] PUT /identity/v3/projects/d011c2789975455ea85b49909d4231cb/users/ea3dad37f03b4dfe9f1019f391914cbc/roles/8feebc60363c4feb870718b1721e5324 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:25:54.936964 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7abe167-7a0b-4e3c-a80a-6237ede7cf7e None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:54.937499 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7abe167-7a0b-4e3c-a80a-6237ede7cf7e None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:54.937499 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7abe167-7a0b-4e3c-a80a-6237ede7cf7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:54.974391 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-a7abe167-7a0b-4e3c-a80a-6237ede7cf7e None None] MFA Rules not processed for user `ea3dad37f03b4dfe9f1019f391914cbc`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:25:54.975832 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-a7abe167-7a0b-4e3c-a80a-6237ede7cf7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:55.049291 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 893/1816] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:25:54 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:25:55.054016 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e59da2-3f0d-43d5-b8d8-10d48cf0d923 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:55.129394 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e59da2-3f0d-43d5-b8d8-10d48cf0d923 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:55.134096 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf448e1-ab9a-492a-b3dd-20e6f995ab3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:55.134732 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf448e1-ab9a-492a-b3dd-20e6f995ab3e admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:55.135021 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf448e1-ab9a-492a-b3dd-20e6f995ab3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:55.135460 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf448e1-ab9a-492a-b3dd-20e6f995ab3e admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:55.136144 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf448e1-ab9a-492a-b3dd-20e6f995ab3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:55.138102 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf448e1-ab9a-492a-b3dd-20e6f995ab3e admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:55.253528 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 924/1817] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:25:55 2022] POST /identity/v3/projects => generated 369 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:25:55.257634 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7abe167-7a0b-4e3c-a80a-6237ede7cf7e None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:55.272513 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7abe167-7a0b-4e3c-a80a-6237ede7cf7e None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:55.349597 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ae2e05-1e27-46c4-a548-83e839e1982a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:55.350246 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ae2e05-1e27-46c4-a548-83e839e1982a admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:55.350618 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ae2e05-1e27-46c4-a548-83e839e1982a admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:55.350902 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ae2e05-1e27-46c4-a548-83e839e1982a admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:55.351544 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ae2e05-1e27-46c4-a548-83e839e1982a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:55.353509 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ae2e05-1e27-46c4-a548-83e839e1982a admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:55.452409 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 894/1818] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:25:55 2022] POST /identity/v3/users => generated 291 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:25:55.457276 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf448e1-ab9a-492a-b3dd-20e6f995ab3e admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:55.479547 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf448e1-ab9a-492a-b3dd-20e6f995ab3e admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:55.485153 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2158ae-1b64-4989-9771-9f43c949616f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:55.486345 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2158ae-1b64-4989-9771-9f43c949616f admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:55.486837 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2158ae-1b64-4989-9771-9f43c949616f admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:55.487360 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2158ae-1b64-4989-9771-9f43c949616f admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:55.488497 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2158ae-1b64-4989-9771-9f43c949616f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:55.490647 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2158ae-1b64-4989-9771-9f43c949616f admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:55.509283 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 925/1819] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:25:55 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:25:55.514247 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ae2e05-1e27-46c4-a548-83e839e1982a admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:55.536389 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ae2e05-1e27-46c4-a548-83e839e1982a admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:55.543693 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a385761e-e81b-44da-b71f-a65d488d6507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:55.544536 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a385761e-e81b-44da-b71f-a65d488d6507 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:55.545453 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a385761e-e81b-44da-b71f-a65d488d6507 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:55.545684 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a385761e-e81b-44da-b71f-a65d488d6507 admin admin] PATH_INFO: `/v3/projects/dc7d53452b3a4ce696a4969a8c296ea5/users/138bdfa1d6f64f2795f67bdc5842c872/roles/a432dd4735944ecfab2fa5ce1705d356` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:55.576534 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a385761e-e81b-44da-b71f-a65d488d6507 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc7d53452b3a4ce696a4969a8c296ea5, user_id=138bdfa1d6f64f2795f67bdc5842c872, role_id=a432dd4735944ecfab2fa5ce1705d356)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:55.580408 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a385761e-e81b-44da-b71f-a65d488d6507 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:55.730417 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2158ae-1b64-4989-9771-9f43c949616f admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:55.735851 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 895/1820] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:25:55 2022] PUT /identity/v3/projects/dc7d53452b3a4ce696a4969a8c296ea5/users/138bdfa1d6f64f2795f67bdc5842c872/roles/a432dd4735944ecfab2fa5ce1705d356 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:25:55.743840 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2158ae-1b64-4989-9771-9f43c949616f admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:55.878037 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ae6dbb-b182-4b6c-9ad8-0559a17a4893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:55.878587 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ae6dbb-b182-4b6c-9ad8-0559a17a4893 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:55.878870 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ae6dbb-b182-4b6c-9ad8-0559a17a4893 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:55.879147 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ae6dbb-b182-4b6c-9ad8-0559a17a4893 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:55.880054 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ae6dbb-b182-4b6c-9ad8-0559a17a4893 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:55.881766 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ae6dbb-b182-4b6c-9ad8-0559a17a4893 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:55.895768 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 926/1821] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:25:55 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:25:55.949557 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a385761e-e81b-44da-b71f-a65d488d6507 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:55.949557 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a385761e-e81b-44da-b71f-a65d488d6507 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:55.949557 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d703c996-f028-47f4-b4fe-2bb60b18c517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:55.949557 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d703c996-f028-47f4-b4fe-2bb60b18c517 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:55.952191 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d703c996-f028-47f4-b4fe-2bb60b18c517 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:55.952191 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d703c996-f028-47f4-b4fe-2bb60b18c517 admin admin] PATH_INFO: `/v3/projects/dc7d53452b3a4ce696a4969a8c296ea5/users/138bdfa1d6f64f2795f67bdc5842c872/roles/a432dd4735944ecfab2fa5ce1705d356` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:55.952191 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d703c996-f028-47f4-b4fe-2bb60b18c517 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc7d53452b3a4ce696a4969a8c296ea5, user_id=138bdfa1d6f64f2795f67bdc5842c872, role_id=a432dd4735944ecfab2fa5ce1705d356)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:55.952191 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d703c996-f028-47f4-b4fe-2bb60b18c517 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:55.952191 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 896/1822] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:25:55 2022] PUT /identity/v3/projects/dc7d53452b3a4ce696a4969a8c296ea5/users/138bdfa1d6f64f2795f67bdc5842c872/roles/a432dd4735944ecfab2fa5ce1705d356 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:25:55.955375 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37606586-19af-47e3-8564-432a207d8d55 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:55.955795 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37606586-19af-47e3-8564-432a207d8d55 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:55.956124 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37606586-19af-47e3-8564-432a207d8d55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:56.012066 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-37606586-19af-47e3-8564-432a207d8d55 None None] MFA Rules not processed for user `138bdfa1d6f64f2795f67bdc5842c872`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:25:56.012066 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-37606586-19af-47e3-8564-432a207d8d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:56.112768 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 927/1823] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:25:55 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:25:56.140891 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d703c996-f028-47f4-b4fe-2bb60b18c517 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:56.143476 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-d703c996-f028-47f4-b4fe-2bb60b18c517 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:56.238952 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:56.273521 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d703c996-f028-47f4-b4fe-2bb60b18c517 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:56.283131 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3426e20e-6cff-4155-85e9-03d70e667a42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e31f2728056b460fa474478eb85ca1d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:56.284761 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3426e20e-6cff-4155-85e9-03d70e667a42 service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:56.356785 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3426e20e-6cff-4155-85e9-03d70e667a42 service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:56.356785 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3426e20e-6cff-4155-85e9-03d70e667a42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:56.356785 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3426e20e-6cff-4155-85e9-03d70e667a42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:56.356785 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3426e20e-6cff-4155-85e9-03d70e667a42 service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:56.356785 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 897/1824] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:25:56 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:25:57.776211 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:58.823281 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:59.363251 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:59.971950 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37606586-19af-47e3-8564-432a207d8d55 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:59.973705 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-37606586-19af-47e3-8564-432a207d8d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:00.067547 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37606586-19af-47e3-8564-432a207d8d55 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:00.072317 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4744e7a5-53d7-4d3f-a402-53cb41c74cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:00.073115 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4744e7a5-53d7-4d3f-a402-53cb41c74cd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:00.073489 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4744e7a5-53d7-4d3f-a402-53cb41c74cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:00.073770 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4744e7a5-53d7-4d3f-a402-53cb41c74cd7 admin admin] PATH_INFO: `/v3/users/35505e9eab184580a2150babd6c33900` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:00.075097 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4744e7a5-53d7-4d3f-a402-53cb41c74cd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35505e9eab184580a2150babd6c33900)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:00.077861 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4744e7a5-53d7-4d3f-a402-53cb41c74cd7 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:00.357481 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-4744e7a5-53d7-4d3f-a402-53cb41c74cd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35505e9eab184580a2150babd6c33900'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:00.373549 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-4744e7a5-53d7-4d3f-a402-53cb41c74cd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35505e9eab184580a2150babd6c33900'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:00.373549 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-4744e7a5-53d7-4d3f-a402-53cb41c74cd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35505e9eab184580a2150babd6c33900'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:00.373549 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-4744e7a5-53d7-4d3f-a402-53cb41c74cd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35505e9eab184580a2150babd6c33900'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:00.388050 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3426e20e-6cff-4155-85e9-03d70e667a42 service cinder] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:00.399144 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3426e20e-6cff-4155-85e9-03d70e667a42 service cinder] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:00.404336 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb98aaff-c64c-492f-91c2-5df75cfcfc8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:00.405077 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb98aaff-c64c-492f-91c2-5df75cfcfc8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:00.405363 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb98aaff-c64c-492f-91c2-5df75cfcfc8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:00.405637 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb98aaff-c64c-492f-91c2-5df75cfcfc8f admin admin] PATH_INFO: `/v3/users/ea3dad37f03b4dfe9f1019f391914cbc` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:00.406938 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb98aaff-c64c-492f-91c2-5df75cfcfc8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea3dad37f03b4dfe9f1019f391914cbc)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:00.409453 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb98aaff-c64c-492f-91c2-5df75cfcfc8f admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:00.456041 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-4744e7a5-53d7-4d3f-a402-53cb41c74cd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35505e9eab184580a2150babd6c33900'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:00.471286 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 928/1825] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:25:59 2022] DELETE /identity/v3/users/35505e9eab184580a2150babd6c33900 => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:26:00.475638 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705f051b-cdda-43ca-abe8-acb242a154bc None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:00.475929 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705f051b-cdda-43ca-abe8-acb242a154bc None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:00.476391 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705f051b-cdda-43ca-abe8-acb242a154bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:00.497611 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-705f051b-cdda-43ca-abe8-acb242a154bc None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:26:00.499033 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-705f051b-cdda-43ca-abe8-acb242a154bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:00.553262 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:26:00.580981 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 929/1826] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:26:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:26:00.596976 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705f051b-cdda-43ca-abe8-acb242a154bc None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:00.598469 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-705f051b-cdda-43ca-abe8-acb242a154bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:00.662343 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-fb98aaff-c64c-492f-91c2-5df75cfcfc8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea3dad37f03b4dfe9f1019f391914cbc'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:00.672569 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705f051b-cdda-43ca-abe8-acb242a154bc None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:00.677005 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910613df-3672-4a32-ba33-7b4cc2a18b1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:00.677378 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910613df-3672-4a32-ba33-7b4cc2a18b1a service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:00.677665 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910613df-3672-4a32-ba33-7b4cc2a18b1a service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:00.677982 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910613df-3672-4a32-ba33-7b4cc2a18b1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:00.687784 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-fb98aaff-c64c-492f-91c2-5df75cfcfc8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea3dad37f03b4dfe9f1019f391914cbc'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:00.691782 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910613df-3672-4a32-ba33-7b4cc2a18b1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:00.692036 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-fb98aaff-c64c-492f-91c2-5df75cfcfc8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea3dad37f03b4dfe9f1019f391914cbc'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:00.693413 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910613df-3672-4a32-ba33-7b4cc2a18b1a service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:00.701378 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-fb98aaff-c64c-492f-91c2-5df75cfcfc8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea3dad37f03b4dfe9f1019f391914cbc'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:00.702322 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-fb98aaff-c64c-492f-91c2-5df75cfcfc8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea3dad37f03b4dfe9f1019f391914cbc'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:00.709300 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 930/1827] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:26:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:26:00.812070 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 898/1828] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:26:00 2022] DELETE /identity/v3/users/ea3dad37f03b4dfe9f1019f391914cbc => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:26:00.814989 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3ae7f4-53d0-490a-b398-c30f4b9f1fbb None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:00.814989 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3ae7f4-53d0-490a-b398-c30f4b9f1fbb None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:00.815340 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3ae7f4-53d0-490a-b398-c30f4b9f1fbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:00.860509 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-dd3ae7f4-53d0-490a-b398-c30f4b9f1fbb None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:26:00.861955 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-dd3ae7f4-53d0-490a-b398-c30f4b9f1fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:00.954021 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 931/1829] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:26:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:26:00.963643 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb98aaff-c64c-492f-91c2-5df75cfcfc8f admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:00.966065 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-fb98aaff-c64c-492f-91c2-5df75cfcfc8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:01.018773 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:26:01.067583 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb98aaff-c64c-492f-91c2-5df75cfcfc8f admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:01.077052 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078cab7a-b10b-40f7-8e56-ea7b8954ac6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:01.077052 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078cab7a-b10b-40f7-8e56-ea7b8954ac6c service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:01.078745 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078cab7a-b10b-40f7-8e56-ea7b8954ac6c service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:01.079113 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078cab7a-b10b-40f7-8e56-ea7b8954ac6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:01.105041 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078cab7a-b10b-40f7-8e56-ea7b8954ac6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:01.116290 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078cab7a-b10b-40f7-8e56-ea7b8954ac6c service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:01.143944 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 899/1830] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:26:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:26:01.576494 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3ae7f4-53d0-490a-b398-c30f4b9f1fbb None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:01.578064 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-dd3ae7f4-53d0-490a-b398-c30f4b9f1fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:01.595089 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3ae7f4-53d0-490a-b398-c30f4b9f1fbb None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:01.609779 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b99838-79f5-491a-bda4-aad8419a99b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:01.615695 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b99838-79f5-491a-bda4-aad8419a99b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:01.615997 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b99838-79f5-491a-bda4-aad8419a99b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:01.624612 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b99838-79f5-491a-bda4-aad8419a99b9 admin admin] PATH_INFO: `/v3/projects/2aef5adc9b204616a7c18ac6cda47710` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:01.624612 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b99838-79f5-491a-bda4-aad8419a99b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2aef5adc9b204616a7c18ac6cda47710)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:01.624612 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b99838-79f5-491a-bda4-aad8419a99b9 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:01.761176 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078cab7a-b10b-40f7-8e56-ea7b8954ac6c service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:01.762761 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-078cab7a-b10b-40f7-8e56-ea7b8954ac6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:01.784494 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078cab7a-b10b-40f7-8e56-ea7b8954ac6c service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:01.796924 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-39b99838-79f5-491a-bda4-aad8419a99b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2aef5adc9b204616a7c18ac6cda47710', 'request_id': 'req-39b99838-79f5-491a-bda4-aad8419a99b9'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:01.802460 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-39b99838-79f5-491a-bda4-aad8419a99b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2aef5adc9b204616a7c18ac6cda47710', 'request_id': 'req-39b99838-79f5-491a-bda4-aad8419a99b9'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:01.872048 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc80d33b-17d0-429f-a588-20ac7c9ba901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:01.873390 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc80d33b-17d0-429f-a588-20ac7c9ba901 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:01.873875 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc80d33b-17d0-429f-a588-20ac7c9ba901 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:01.874301 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc80d33b-17d0-429f-a588-20ac7c9ba901 admin admin] PATH_INFO: `/v3/projects/d011c2789975455ea85b49909d4231cb` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:01.877226 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc80d33b-17d0-429f-a588-20ac7c9ba901 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d011c2789975455ea85b49909d4231cb)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:01.879122 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-39b99838-79f5-491a-bda4-aad8419a99b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2aef5adc9b204616a7c18ac6cda47710', 'request_id': 'req-39b99838-79f5-491a-bda4-aad8419a99b9'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:01.880054 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc80d33b-17d0-429f-a588-20ac7c9ba901 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:01.919978 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-39b99838-79f5-491a-bda4-aad8419a99b9 admin admin] The token cache is being invalidate because project 2aef5adc9b204616a7c18ac6cda47710 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:26:01.923385 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-39b99838-79f5-491a-bda4-aad8419a99b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:01.928783 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-39b99838-79f5-491a-bda4-aad8419a99b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:01.936033 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 932/1831] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:26:01 2022] DELETE /identity/v3/projects/2aef5adc9b204616a7c18ac6cda47710 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:26:01.942705 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b99838-79f5-491a-bda4-aad8419a99b9 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:01.945230 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-39b99838-79f5-491a-bda4-aad8419a99b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:01.973426 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b99838-79f5-491a-bda4-aad8419a99b9 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:01.980911 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74f585c-a858-4512-88e0-6114aa0e5db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:01.981847 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74f585c-a858-4512-88e0-6114aa0e5db0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:01.982282 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74f585c-a858-4512-88e0-6114aa0e5db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:01.982731 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74f585c-a858-4512-88e0-6114aa0e5db0 admin admin] PATH_INFO: `/v3/users/0fe596f1c0d14f3f9e02e890f1bee050` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:01.984324 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74f585c-a858-4512-88e0-6114aa0e5db0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fe596f1c0d14f3f9e02e890f1bee050)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:01.987182 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74f585c-a858-4512-88e0-6114aa0e5db0 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:02.113564 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-bc80d33b-17d0-429f-a588-20ac7c9ba901 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd011c2789975455ea85b49909d4231cb', 'request_id': 'req-bc80d33b-17d0-429f-a588-20ac7c9ba901'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:02.125109 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-bc80d33b-17d0-429f-a588-20ac7c9ba901 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd011c2789975455ea85b49909d4231cb', 'request_id': 'req-bc80d33b-17d0-429f-a588-20ac7c9ba901'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:02.212806 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-c74f585c-a858-4512-88e0-6114aa0e5db0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fe596f1c0d14f3f9e02e890f1bee050'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:02.227797 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:26:02.234222 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-bc80d33b-17d0-429f-a588-20ac7c9ba901 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd011c2789975455ea85b49909d4231cb', 'request_id': 'req-bc80d33b-17d0-429f-a588-20ac7c9ba901'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:02.246575 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-c74f585c-a858-4512-88e0-6114aa0e5db0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fe596f1c0d14f3f9e02e890f1bee050'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:02.247004 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-c74f585c-a858-4512-88e0-6114aa0e5db0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fe596f1c0d14f3f9e02e890f1bee050'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:02.248332 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-c74f585c-a858-4512-88e0-6114aa0e5db0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fe596f1c0d14f3f9e02e890f1bee050'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:02.249713 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-bc80d33b-17d0-429f-a588-20ac7c9ba901 admin admin] The token cache is being invalidate because project d011c2789975455ea85b49909d4231cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:26:02.250396 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-bc80d33b-17d0-429f-a588-20ac7c9ba901 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:02.252845 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-bc80d33b-17d0-429f-a588-20ac7c9ba901 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:02.258161 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 900/1832] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:26:01 2022] DELETE /identity/v3/projects/d011c2789975455ea85b49909d4231cb => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:26:02.261399 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc80d33b-17d0-429f-a588-20ac7c9ba901 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:02.268644 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-bc80d33b-17d0-429f-a588-20ac7c9ba901 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:02.346450 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc80d33b-17d0-429f-a588-20ac7c9ba901 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:02.346909 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-c74f585c-a858-4512-88e0-6114aa0e5db0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fe596f1c0d14f3f9e02e890f1bee050'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:02.356294 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cae4dd-ea67-4aba-9fb0-d5cdb5d36ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:02.359131 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cae4dd-ea67-4aba-9fb0-d5cdb5d36ad5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:02.359482 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cae4dd-ea67-4aba-9fb0-d5cdb5d36ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:02.359768 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cae4dd-ea67-4aba-9fb0-d5cdb5d36ad5 admin admin] PATH_INFO: `/v3/users/138bdfa1d6f64f2795f67bdc5842c872` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:02.361319 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cae4dd-ea67-4aba-9fb0-d5cdb5d36ad5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=138bdfa1d6f64f2795f67bdc5842c872)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:02.363026 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 933/1833] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:26:01 2022] DELETE /identity/v3/users/0fe596f1c0d14f3f9e02e890f1bee050 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:26:02.363470 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cae4dd-ea67-4aba-9fb0-d5cdb5d36ad5 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:02.575008 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-16cae4dd-ea67-4aba-9fb0-d5cdb5d36ad5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '138bdfa1d6f64f2795f67bdc5842c872'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:02.587890 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:26:02.598850 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-16cae4dd-ea67-4aba-9fb0-d5cdb5d36ad5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '138bdfa1d6f64f2795f67bdc5842c872'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:02.599644 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-16cae4dd-ea67-4aba-9fb0-d5cdb5d36ad5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '138bdfa1d6f64f2795f67bdc5842c872'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:02.607304 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-16cae4dd-ea67-4aba-9fb0-d5cdb5d36ad5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '138bdfa1d6f64f2795f67bdc5842c872'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:02.608023 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-16cae4dd-ea67-4aba-9fb0-d5cdb5d36ad5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '138bdfa1d6f64f2795f67bdc5842c872'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:02.732480 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 901/1834] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:26:02 2022] DELETE /identity/v3/users/138bdfa1d6f64f2795f67bdc5842c872 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:26:03.064462 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:26:03.143473 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74f585c-a858-4512-88e0-6114aa0e5db0 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:03.145046 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-c74f585c-a858-4512-88e0-6114aa0e5db0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:03.333445 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74f585c-a858-4512-88e0-6114aa0e5db0 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:03.420015 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adda0dc2-af5f-48f3-91c7-eaecb6a2805b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:03.420764 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adda0dc2-af5f-48f3-91c7-eaecb6a2805b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:03.420961 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adda0dc2-af5f-48f3-91c7-eaecb6a2805b admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:03.421283 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adda0dc2-af5f-48f3-91c7-eaecb6a2805b admin admin] PATH_INFO: `/v3/projects/db7d7ca21538415a979dffc032b6ddd1` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:03.423206 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adda0dc2-af5f-48f3-91c7-eaecb6a2805b admin admin] RBAC: Authorizing `identity:delete_project(project_id=db7d7ca21538415a979dffc032b6ddd1)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:03.425157 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adda0dc2-af5f-48f3-91c7-eaecb6a2805b admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:03.520453 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cae4dd-ea67-4aba-9fb0-d5cdb5d36ad5 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:03.523129 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-16cae4dd-ea67-4aba-9fb0-d5cdb5d36ad5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:03.572061 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cae4dd-ea67-4aba-9fb0-d5cdb5d36ad5 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:03.593145 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-adda0dc2-af5f-48f3-91c7-eaecb6a2805b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db7d7ca21538415a979dffc032b6ddd1', 'request_id': 'req-adda0dc2-af5f-48f3-91c7-eaecb6a2805b'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:03.604900 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-adda0dc2-af5f-48f3-91c7-eaecb6a2805b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db7d7ca21538415a979dffc032b6ddd1', 'request_id': 'req-adda0dc2-af5f-48f3-91c7-eaecb6a2805b'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:03.666772 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7853927a-2719-415f-aa13-82e335ab85f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:03.667496 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7853927a-2719-415f-aa13-82e335ab85f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:03.667776 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7853927a-2719-415f-aa13-82e335ab85f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:03.668050 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7853927a-2719-415f-aa13-82e335ab85f7 admin admin] PATH_INFO: `/v3/projects/dc7d53452b3a4ce696a4969a8c296ea5` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:03.669232 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7853927a-2719-415f-aa13-82e335ab85f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc7d53452b3a4ce696a4969a8c296ea5)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:03.671228 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7853927a-2719-415f-aa13-82e335ab85f7 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:03.701154 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-adda0dc2-af5f-48f3-91c7-eaecb6a2805b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db7d7ca21538415a979dffc032b6ddd1', 'request_id': 'req-adda0dc2-af5f-48f3-91c7-eaecb6a2805b'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:03.716320 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-adda0dc2-af5f-48f3-91c7-eaecb6a2805b admin admin] The token cache is being invalidate because project db7d7ca21538415a979dffc032b6ddd1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:26:03.716904 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-adda0dc2-af5f-48f3-91c7-eaecb6a2805b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:03.718539 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-adda0dc2-af5f-48f3-91c7-eaecb6a2805b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:03.720906 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 934/1835] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:26:03 2022] DELETE /identity/v3/projects/db7d7ca21538415a979dffc032b6ddd1 => generated 0 bytes in 578 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:26:03.725805 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7850d07-924f-4525-8cc9-8a379eb528b9 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:03.726277 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7850d07-924f-4525-8cc9-8a379eb528b9 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:03.726708 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7850d07-924f-4525-8cc9-8a379eb528b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:03.750448 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-b7850d07-924f-4525-8cc9-8a379eb528b9 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:26:03.751807 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-b7850d07-924f-4525-8cc9-8a379eb528b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:03.760104 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 935/1836] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:26:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:26:03.764681 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=80,keystone:INSERT=3,keystone:DELETE=17 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:26:03.768156 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fe494c-5991-45e6-bf04-176fdef9360d None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:03.768444 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fe494c-5991-45e6-bf04-176fdef9360d None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:03.768823 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fe494c-5991-45e6-bf04-176fdef9360d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:03.790455 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-08fe494c-5991-45e6-bf04-176fdef9360d None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:26:03.800316 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-08fe494c-5991-45e6-bf04-176fdef9360d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:03.829355 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-7853927a-2719-415f-aa13-82e335ab85f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc7d53452b3a4ce696a4969a8c296ea5', 'request_id': 'req-7853927a-2719-415f-aa13-82e335ab85f7'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:03.836690 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-7853927a-2719-415f-aa13-82e335ab85f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc7d53452b3a4ce696a4969a8c296ea5', 'request_id': 'req-7853927a-2719-415f-aa13-82e335ab85f7'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:03.890179 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 936/1837] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:26:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:26:03.896183 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fe494c-5991-45e6-bf04-176fdef9360d None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:03.908197 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fe494c-5991-45e6-bf04-176fdef9360d None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:03.913873 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fd6113-cd85-41a7-97af-0f3cae3ec3c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:03.914685 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fd6113-cd85-41a7-97af-0f3cae3ec3c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:03.915140 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fd6113-cd85-41a7-97af-0f3cae3ec3c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:03.915506 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fd6113-cd85-41a7-97af-0f3cae3ec3c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:03.916340 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fd6113-cd85-41a7-97af-0f3cae3ec3c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:26:03.916736 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fd6113-cd85-41a7-97af-0f3cae3ec3c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:03.918662 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fd6113-cd85-41a7-97af-0f3cae3ec3c4 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:03.934848 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 937/1838] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Aug 4 21:26:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:26:03.936000 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fd6113-cd85-41a7-97af-0f3cae3ec3c4 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:03.937557 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-7853927a-2719-415f-aa13-82e335ab85f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc7d53452b3a4ce696a4969a8c296ea5', 'request_id': 'req-7853927a-2719-415f-aa13-82e335ab85f7'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:03.946396 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fd6113-cd85-41a7-97af-0f3cae3ec3c4 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:03.951561 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-7853927a-2719-415f-aa13-82e335ab85f7 admin admin] The token cache is being invalidate because project dc7d53452b3a4ce696a4969a8c296ea5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:26:03.951891 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-7853927a-2719-415f-aa13-82e335ab85f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:03.952468 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1916b5-034b-4004-8c5c-6efc1b86bbac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:03.952595 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-7853927a-2719-415f-aa13-82e335ab85f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:03.953288 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1916b5-034b-4004-8c5c-6efc1b86bbac admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:03.953649 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1916b5-034b-4004-8c5c-6efc1b86bbac admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:03.954541 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1916b5-034b-4004-8c5c-6efc1b86bbac admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:03.955190 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1916b5-034b-4004-8c5c-6efc1b86bbac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:03.956590 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 902/1839] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:26:03 2022] DELETE /identity/v3/projects/dc7d53452b3a4ce696a4969a8c296ea5 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:26:03.957345 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1916b5-034b-4004-8c5c-6efc1b86bbac admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:03.964020 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef92e76b-23c2-4288-8ed9-a711f9d3025d None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:03.964308 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef92e76b-23c2-4288-8ed9-a711f9d3025d None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:03.964586 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef92e76b-23c2-4288-8ed9-a711f9d3025d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:03.985262 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-ef92e76b-23c2-4288-8ed9-a711f9d3025d None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:26:03.990809 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-ef92e76b-23c2-4288-8ed9-a711f9d3025d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:03.994768 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 903/1840] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:26:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:26:04.002519 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df9b9bf-8535-4626-87b4-bc17d52024eb None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:04.002803 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df9b9bf-8535-4626-87b4-bc17d52024eb None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:04.003087 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df9b9bf-8535-4626-87b4-bc17d52024eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:04.023290 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-8df9b9bf-8535-4626-87b4-bc17d52024eb None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:26:04.024657 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-8df9b9bf-8535-4626-87b4-bc17d52024eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:04.034876 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 904/1841] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:26:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:26:04.043758 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df9b9bf-8535-4626-87b4-bc17d52024eb None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:04.053335 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df9b9bf-8535-4626-87b4-bc17d52024eb None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:04.058069 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce54ea6-3b17-4f7d-8827-236c95dc6cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:04.059713 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce54ea6-3b17-4f7d-8827-236c95dc6cc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:04.059713 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce54ea6-3b17-4f7d-8827-236c95dc6cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:04.059713 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce54ea6-3b17-4f7d-8827-236c95dc6cc8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:04.060015 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce54ea6-3b17-4f7d-8827-236c95dc6cc8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:26:04.060199 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce54ea6-3b17-4f7d-8827-236c95dc6cc8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:04.061811 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce54ea6-3b17-4f7d-8827-236c95dc6cc8 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:04.073956 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 905/1842] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Aug 4 21:26:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:26:04.078436 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce54ea6-3b17-4f7d-8827-236c95dc6cc8 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:04.090370 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce54ea6-3b17-4f7d-8827-236c95dc6cc8 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:04.096706 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912d4ad0-42a2-4ede-a7a2-c6801b9f20f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:04.097369 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912d4ad0-42a2-4ede-a7a2-c6801b9f20f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:04.097655 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912d4ad0-42a2-4ede-a7a2-c6801b9f20f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:04.097931 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912d4ad0-42a2-4ede-a7a2-c6801b9f20f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:04.098615 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912d4ad0-42a2-4ede-a7a2-c6801b9f20f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:04.100644 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912d4ad0-42a2-4ede-a7a2-c6801b9f20f8 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:04.126226 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 938/1843] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:26:03 2022] POST /identity/v3/projects => generated 363 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:26:04.131852 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1916b5-034b-4004-8c5c-6efc1b86bbac admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:04.133506 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-3e1916b5-034b-4004-8c5c-6efc1b86bbac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:04.201119 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1916b5-034b-4004-8c5c-6efc1b86bbac admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:04.216436 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 906/1844] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:26:04 2022] POST /identity/v3/projects => generated 387 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:26:04.216625 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74681a82-5aee-4c26-a7ef-9e4ea13775c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:04.216625 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74681a82-5aee-4c26-a7ef-9e4ea13775c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:04.216625 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74681a82-5aee-4c26-a7ef-9e4ea13775c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:04.216625 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74681a82-5aee-4c26-a7ef-9e4ea13775c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:04.216986 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74681a82-5aee-4c26-a7ef-9e4ea13775c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:04.216986 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74681a82-5aee-4c26-a7ef-9e4ea13775c2 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:04.217194 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912d4ad0-42a2-4ede-a7a2-c6801b9f20f8 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:04.310812 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912d4ad0-42a2-4ede-a7a2-c6801b9f20f8 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:04.320078 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f90946-c076-439e-8286-9d3b249484ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:04.320757 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f90946-c076-439e-8286-9d3b249484ea admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:04.321044 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f90946-c076-439e-8286-9d3b249484ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:04.321328 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f90946-c076-439e-8286-9d3b249484ea admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:04.321997 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f90946-c076-439e-8286-9d3b249484ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:04.324096 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f90946-c076-439e-8286-9d3b249484ea admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:04.479227 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 939/1845] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:26:04 2022] POST /identity/v3/users => generated 282 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:26:04.484343 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74681a82-5aee-4c26-a7ef-9e4ea13775c2 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:04.498926 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74681a82-5aee-4c26-a7ef-9e4ea13775c2 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:04.509273 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0ac0cc-2e81-4ddc-b409-3e15245643c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:04.511274 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0ac0cc-2e81-4ddc-b409-3e15245643c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:04.511274 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0ac0cc-2e81-4ddc-b409-3e15245643c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:04.511274 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0ac0cc-2e81-4ddc-b409-3e15245643c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:04.511611 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0ac0cc-2e81-4ddc-b409-3e15245643c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:04.513045 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0ac0cc-2e81-4ddc-b409-3e15245643c8 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:04.517439 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 907/1846] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:26:04 2022] POST /identity/v3/users => generated 294 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:26:04.523026 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f90946-c076-439e-8286-9d3b249484ea admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:04.526246 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 940/1847] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:26:04 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:26:04.529986 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0ac0cc-2e81-4ddc-b409-3e15245643c8 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:04.539305 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f90946-c076-439e-8286-9d3b249484ea admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:04.540561 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0ac0cc-2e81-4ddc-b409-3e15245643c8 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:04.545495 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c52e705-1e42-46fa-a02d-f84f8884aa37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:04.546145 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c52e705-1e42-46fa-a02d-f84f8884aa37 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:04.546474 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f643119c-581c-4af6-b690-102f5bb3e32c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:04.546640 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c52e705-1e42-46fa-a02d-f84f8884aa37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:04.546872 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c52e705-1e42-46fa-a02d-f84f8884aa37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:04.548272 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c52e705-1e42-46fa-a02d-f84f8884aa37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:04.549074 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f643119c-581c-4af6-b690-102f5bb3e32c admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:04.549564 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f643119c-581c-4af6-b690-102f5bb3e32c admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:04.549915 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c52e705-1e42-46fa-a02d-f84f8884aa37 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:04.550147 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f643119c-581c-4af6-b690-102f5bb3e32c admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:04.551129 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f643119c-581c-4af6-b690-102f5bb3e32c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:04.554051 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f643119c-581c-4af6-b690-102f5bb3e32c admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:04.577773 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 941/1848] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:26:04 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:26:04.577773 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c52e705-1e42-46fa-a02d-f84f8884aa37 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:04.585006 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 908/1849] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:26:04 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:26:04.589867 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c52e705-1e42-46fa-a02d-f84f8884aa37 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:04.591173 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f643119c-581c-4af6-b690-102f5bb3e32c admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:04.597918 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644496e0-b8da-43ba-b12e-6459f93660fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:04.598859 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644496e0-b8da-43ba-b12e-6459f93660fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:04.599232 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644496e0-b8da-43ba-b12e-6459f93660fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:04.599656 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644496e0-b8da-43ba-b12e-6459f93660fb admin admin] PATH_INFO: `/v3/projects/48c6cf8ce81d4fe59668a421906b751c/users/4f0830ef216244f0a73a8f164ad26a31/roles/8feebc60363c4feb870718b1721e5324` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:04.612113 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f643119c-581c-4af6-b690-102f5bb3e32c admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:04.618122 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b3816d-ca2d-45f3-a011-285044fcd6d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:04.619130 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b3816d-ca2d-45f3-a011-285044fcd6d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:04.619461 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b3816d-ca2d-45f3-a011-285044fcd6d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:04.619744 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b3816d-ca2d-45f3-a011-285044fcd6d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:04.620645 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b3816d-ca2d-45f3-a011-285044fcd6d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:04.621178 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644496e0-b8da-43ba-b12e-6459f93660fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=48c6cf8ce81d4fe59668a421906b751c, user_id=4f0830ef216244f0a73a8f164ad26a31, role_id=8feebc60363c4feb870718b1721e5324)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:04.622492 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b3816d-ca2d-45f3-a011-285044fcd6d7 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:04.624579 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644496e0-b8da-43ba-b12e-6459f93660fb admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:04.640729 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 909/1850] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:26:04 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:26:04.641881 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b3816d-ca2d-45f3-a011-285044fcd6d7 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:04.656568 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b3816d-ca2d-45f3-a011-285044fcd6d7 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:04.671067 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5123a71d-9b32-4da8-970b-2c23f06f3ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:04.672237 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5123a71d-9b32-4da8-970b-2c23f06f3ee1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:04.673276 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5123a71d-9b32-4da8-970b-2c23f06f3ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:04.673863 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5123a71d-9b32-4da8-970b-2c23f06f3ee1 admin admin] PATH_INFO: `/v3/projects/b51a1b54a7554b408dfaaf90e0337156/users/32ed30b6d278486ab1cd2eb6e352e344/roles/8feebc60363c4feb870718b1721e5324` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:04.691443 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5123a71d-9b32-4da8-970b-2c23f06f3ee1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b51a1b54a7554b408dfaaf90e0337156, user_id=32ed30b6d278486ab1cd2eb6e352e344, role_id=8feebc60363c4feb870718b1721e5324)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:04.694575 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5123a71d-9b32-4da8-970b-2c23f06f3ee1 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:04.757246 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 942/1851] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:26:04 2022] PUT /identity/v3/projects/48c6cf8ce81d4fe59668a421906b751c/users/4f0830ef216244f0a73a8f164ad26a31/roles/8feebc60363c4feb870718b1721e5324 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:26:04.774880 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eebf538-3b79-4054-b733-36ce124d4a57 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:04.775653 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eebf538-3b79-4054-b733-36ce124d4a57 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:04.776187 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eebf538-3b79-4054-b733-36ce124d4a57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:04.800634 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 910/1852] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:26:04 2022] PUT /identity/v3/projects/b51a1b54a7554b408dfaaf90e0337156/users/32ed30b6d278486ab1cd2eb6e352e344/roles/8feebc60363c4feb870718b1721e5324 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:26:04.817491 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c0503c-acb9-49c0-aa68-881ba2a27db9 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:04.817772 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c0503c-acb9-49c0-aa68-881ba2a27db9 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:04.818051 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c0503c-acb9-49c0-aa68-881ba2a27db9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:04.837399 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-1eebf538-3b79-4054-b733-36ce124d4a57 None None] MFA Rules not processed for user `4f0830ef216244f0a73a8f164ad26a31`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:26:04.839113 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-1eebf538-3b79-4054-b733-36ce124d4a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:04.852721 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-59c0503c-acb9-49c0-aa68-881ba2a27db9 None None] MFA Rules not processed for user `32ed30b6d278486ab1cd2eb6e352e344`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:26:04.854728 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-59c0503c-acb9-49c0-aa68-881ba2a27db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:04.975466 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 943/1853] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:26:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:26:04.977488 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 911/1854] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:26:04 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:26:04.980390 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eebf538-3b79-4054-b733-36ce124d4a57 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:04.982679 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c0503c-acb9-49c0-aa68-881ba2a27db9 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:05.072997 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:26:05.075333 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eebf538-3b79-4054-b733-36ce124d4a57 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:05.079297 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c0503c-acb9-49c0-aa68-881ba2a27db9 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:05.082132 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511c3fa6-21f4-4d7c-bf43-ae83c801c270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:05.083142 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511c3fa6-21f4-4d7c-bf43-ae83c801c270 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:05.083735 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511c3fa6-21f4-4d7c-bf43-ae83c801c270 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:05.084255 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511c3fa6-21f4-4d7c-bf43-ae83c801c270 admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:05.085239 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511c3fa6-21f4-4d7c-bf43-ae83c801c270 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:05.085783 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daedb44a-9b4a-47b8-9c0b-33096a152465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:05.086490 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daedb44a-9b4a-47b8-9c0b-33096a152465 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:05.086840 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daedb44a-9b4a-47b8-9c0b-33096a152465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:05.087197 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daedb44a-9b4a-47b8-9c0b-33096a152465 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:05.087968 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daedb44a-9b4a-47b8-9c0b-33096a152465 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:05.088483 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511c3fa6-21f4-4d7c-bf43-ae83c801c270 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:05.089973 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daedb44a-9b4a-47b8-9c0b-33096a152465 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:05.190749 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 912/1855] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:26:04 2022] POST /identity/v3/projects => generated 387 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:26:05.195625 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daedb44a-9b4a-47b8-9c0b-33096a152465 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:05.200977 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 944/1856] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:26:04 2022] POST /identity/v3/projects => generated 363 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:26:05.202054 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511c3fa6-21f4-4d7c-bf43-ae83c801c270 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:05.286385 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511c3fa6-21f4-4d7c-bf43-ae83c801c270 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:05.290045 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daedb44a-9b4a-47b8-9c0b-33096a152465 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:05.293337 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8a4cb5-c203-45a4-a18d-1b0d8373c9d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:05.294629 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8a4cb5-c203-45a4-a18d-1b0d8373c9d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:05.294913 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8a4cb5-c203-45a4-a18d-1b0d8373c9d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:05.295185 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8a4cb5-c203-45a4-a18d-1b0d8373c9d3 admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:05.295899 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8a4cb5-c203-45a4-a18d-1b0d8373c9d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:05.296356 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827f335b-05d7-4f7c-b611-7c91135bba82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:05.298600 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827f335b-05d7-4f7c-b611-7c91135bba82 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:05.299040 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8a4cb5-c203-45a4-a18d-1b0d8373c9d3 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:05.299442 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827f335b-05d7-4f7c-b611-7c91135bba82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:05.299925 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827f335b-05d7-4f7c-b611-7c91135bba82 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:05.300777 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827f335b-05d7-4f7c-b611-7c91135bba82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:05.302973 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827f335b-05d7-4f7c-b611-7c91135bba82 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:05.396477 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 945/1857] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:26:05 2022] POST /identity/v3/users => generated 288 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:26:05.397436 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8a4cb5-c203-45a4-a18d-1b0d8373c9d3 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:05.398340 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 913/1858] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:26:05 2022] POST /identity/v3/users => generated 300 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:26:05.401677 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827f335b-05d7-4f7c-b611-7c91135bba82 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:05.415250 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8a4cb5-c203-45a4-a18d-1b0d8373c9d3 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:05.424176 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b63b8ad-52e1-44b0-a122-7db048fe0b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:05.426489 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b63b8ad-52e1-44b0-a122-7db048fe0b5a admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:05.427434 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b63b8ad-52e1-44b0-a122-7db048fe0b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:05.427881 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b63b8ad-52e1-44b0-a122-7db048fe0b5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:05.428433 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827f335b-05d7-4f7c-b611-7c91135bba82 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:05.429208 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b63b8ad-52e1-44b0-a122-7db048fe0b5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:05.431112 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b63b8ad-52e1-44b0-a122-7db048fe0b5a admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:05.440572 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00405f36-1e96-421f-814b-db4ca3194a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:05.441460 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00405f36-1e96-421f-814b-db4ca3194a5f admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:05.441749 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00405f36-1e96-421f-814b-db4ca3194a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:05.442029 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00405f36-1e96-421f-814b-db4ca3194a5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:05.442750 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00405f36-1e96-421f-814b-db4ca3194a5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:05.444438 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00405f36-1e96-421f-814b-db4ca3194a5f admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:05.459761 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 946/1859] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:26:05 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:26:05.461544 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b63b8ad-52e1-44b0-a122-7db048fe0b5a admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:05.470017 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 914/1860] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:26:05 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:26:05.485768 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00405f36-1e96-421f-814b-db4ca3194a5f admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:05.496231 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b63b8ad-52e1-44b0-a122-7db048fe0b5a admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:05.503804 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdac08eb-d633-403f-a0f6-e1eeb63b9569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:05.504882 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdac08eb-d633-403f-a0f6-e1eeb63b9569 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:05.505166 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdac08eb-d633-403f-a0f6-e1eeb63b9569 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:05.505560 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdac08eb-d633-403f-a0f6-e1eeb63b9569 admin admin] PATH_INFO: `/v3/projects/0006aad3f9d14cc896663a5c5a558dd0/users/ea0efe59c2974606bb77e6cd0882f113/roles/a432dd4735944ecfab2fa5ce1705d356` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:05.506057 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00405f36-1e96-421f-814b-db4ca3194a5f admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:05.511517 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe59dea2-744a-4952-913d-07b33e346fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:05.512499 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe59dea2-744a-4952-913d-07b33e346fa1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:05.513020 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe59dea2-744a-4952-913d-07b33e346fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:05.513514 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe59dea2-744a-4952-913d-07b33e346fa1 admin admin] PATH_INFO: `/v3/projects/3c7268e7eb3d4a71bcca41919a3b655f/users/b622ea9bfe0546ddb17033ada9ce255a/roles/a432dd4735944ecfab2fa5ce1705d356` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:05.524956 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdac08eb-d633-403f-a0f6-e1eeb63b9569 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0006aad3f9d14cc896663a5c5a558dd0, user_id=ea0efe59c2974606bb77e6cd0882f113, role_id=a432dd4735944ecfab2fa5ce1705d356)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:05.528256 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdac08eb-d633-403f-a0f6-e1eeb63b9569 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:05.535956 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe59dea2-744a-4952-913d-07b33e346fa1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c7268e7eb3d4a71bcca41919a3b655f, user_id=b622ea9bfe0546ddb17033ada9ce255a, role_id=a432dd4735944ecfab2fa5ce1705d356)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:05.539064 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe59dea2-744a-4952-913d-07b33e346fa1 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:05.677144 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 947/1861] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:26:05 2022] PUT /identity/v3/projects/0006aad3f9d14cc896663a5c5a558dd0/users/ea0efe59c2974606bb77e6cd0882f113/roles/a432dd4735944ecfab2fa5ce1705d356 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:26:05.680936 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdac08eb-d633-403f-a0f6-e1eeb63b9569 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:05.747145 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 915/1862] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:26:05 2022] PUT /identity/v3/projects/3c7268e7eb3d4a71bcca41919a3b655f/users/b622ea9bfe0546ddb17033ada9ce255a/roles/a432dd4735944ecfab2fa5ce1705d356 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:26:05.749163 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe59dea2-744a-4952-913d-07b33e346fa1 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:05.783541 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdac08eb-d633-403f-a0f6-e1eeb63b9569 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:05.857190 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe59dea2-744a-4952-913d-07b33e346fa1 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:05.864494 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3879520-313c-449d-a55f-dd10f6e9a811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:05.864494 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3879520-313c-449d-a55f-dd10f6e9a811 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:05.864494 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3879520-313c-449d-a55f-dd10f6e9a811 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:05.864494 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3879520-313c-449d-a55f-dd10f6e9a811 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:05.864950 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3879520-313c-449d-a55f-dd10f6e9a811 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:05.865834 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3879520-313c-449d-a55f-dd10f6e9a811 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:05.868158 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49847a60-6ad5-48fc-93fc-b714fff07d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:05.869412 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49847a60-6ad5-48fc-93fc-b714fff07d91 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:05.869838 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49847a60-6ad5-48fc-93fc-b714fff07d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:05.870063 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49847a60-6ad5-48fc-93fc-b714fff07d91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:05.870925 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49847a60-6ad5-48fc-93fc-b714fff07d91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:05.872726 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49847a60-6ad5-48fc-93fc-b714fff07d91 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:05.881491 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 916/1863] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:26:05 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:26:05.886917 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3879520-313c-449d-a55f-dd10f6e9a811 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:05.893212 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 948/1864] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:26:05 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:26:05.897318 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49847a60-6ad5-48fc-93fc-b714fff07d91 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:05.902924 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3879520-313c-449d-a55f-dd10f6e9a811 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:05.909241 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1617d04a-c795-4e28-a8b8-bc3e19688444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:05.910085 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1617d04a-c795-4e28-a8b8-bc3e19688444 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:05.910903 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1617d04a-c795-4e28-a8b8-bc3e19688444 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:05.910903 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1617d04a-c795-4e28-a8b8-bc3e19688444 admin admin] PATH_INFO: `/v3/projects/3c7268e7eb3d4a71bcca41919a3b655f/users/b622ea9bfe0546ddb17033ada9ce255a/roles/a432dd4735944ecfab2fa5ce1705d356` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:05.912684 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1617d04a-c795-4e28-a8b8-bc3e19688444 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c7268e7eb3d4a71bcca41919a3b655f, user_id=b622ea9bfe0546ddb17033ada9ce255a, role_id=a432dd4735944ecfab2fa5ce1705d356)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:05.915089 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49847a60-6ad5-48fc-93fc-b714fff07d91 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:05.915387 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1617d04a-c795-4e28-a8b8-bc3e19688444 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:05.922662 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b116c736-301d-4667-b926-83629ccd860b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:05.928608 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b116c736-301d-4667-b926-83629ccd860b admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:05.928608 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b116c736-301d-4667-b926-83629ccd860b admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:05.928608 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b116c736-301d-4667-b926-83629ccd860b admin admin] PATH_INFO: `/v3/projects/0006aad3f9d14cc896663a5c5a558dd0/users/ea0efe59c2974606bb77e6cd0882f113/roles/a432dd4735944ecfab2fa5ce1705d356` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:05.928608 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b116c736-301d-4667-b926-83629ccd860b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0006aad3f9d14cc896663a5c5a558dd0, user_id=ea0efe59c2974606bb77e6cd0882f113, role_id=a432dd4735944ecfab2fa5ce1705d356)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:05.940912 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 917/1865] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:26:05 2022] PUT /identity/v3/projects/3c7268e7eb3d4a71bcca41919a3b655f/users/b622ea9bfe0546ddb17033ada9ce255a/roles/a432dd4735944ecfab2fa5ce1705d356 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:26:05.941010 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b116c736-301d-4667-b926-83629ccd860b admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:05.942095 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aaf667-28c6-480e-af15-788d081c57c0 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:05.942095 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aaf667-28c6-480e-af15-788d081c57c0 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:05.942095 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aaf667-28c6-480e-af15-788d081c57c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:05.956446 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 949/1866] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:26:05 2022] PUT /identity/v3/projects/0006aad3f9d14cc896663a5c5a558dd0/users/ea0efe59c2974606bb77e6cd0882f113/roles/a432dd4735944ecfab2fa5ce1705d356 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:26:05.971590 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c225b6e-765b-42c3-a720-ef5c977970da None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:05.972027 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c225b6e-765b-42c3-a720-ef5c977970da None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:05.972223 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c225b6e-765b-42c3-a720-ef5c977970da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:06.010981 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-85aaf667-28c6-480e-af15-788d081c57c0 None None] MFA Rules not processed for user `b622ea9bfe0546ddb17033ada9ce255a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:26:06.012248 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-85aaf667-28c6-480e-af15-788d081c57c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:06.018321 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-5c225b6e-765b-42c3-a720-ef5c977970da None None] MFA Rules not processed for user `ea0efe59c2974606bb77e6cd0882f113`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:26:06.019888 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-5c225b6e-765b-42c3-a720-ef5c977970da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:06.146933 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 950/1867] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:26:05 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:26:06.158928 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c225b6e-765b-42c3-a720-ef5c977970da None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:06.159448 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-5c225b6e-765b-42c3-a720-ef5c977970da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:06.184859 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 918/1868] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:26:05 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:26:06.258495 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c225b6e-765b-42c3-a720-ef5c977970da None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:06.263019 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024ea3b4-5577-4c5c-97a5-1ed83f31a8f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e31f2728056b460fa474478eb85ca1d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:06.263940 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ea3b4-5577-4c5c-97a5-1ed83f31a8f8 service cinder] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:06.264222 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ea3b4-5577-4c5c-97a5-1ed83f31a8f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:06.264489 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ea3b4-5577-4c5c-97a5-1ed83f31a8f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:06.274431 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024ea3b4-5577-4c5c-97a5-1ed83f31a8f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:06.276064 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024ea3b4-5577-4c5c-97a5-1ed83f31a8f8 service cinder] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:06.286579 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 951/1869] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:26:06 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:26:06.287847 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:26:06.312428 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aaf667-28c6-480e-af15-788d081c57c0 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:06.322485 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aaf667-28c6-480e-af15-788d081c57c0 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:06.328091 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce65f4c-4337-4e7a-95c5-be6c92783350 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e31f2728056b460fa474478eb85ca1d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:06.329106 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce65f4c-4337-4e7a-95c5-be6c92783350 service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:06.329522 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce65f4c-4337-4e7a-95c5-be6c92783350 service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:06.329914 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce65f4c-4337-4e7a-95c5-be6c92783350 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:06.340663 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce65f4c-4337-4e7a-95c5-be6c92783350 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:06.342119 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce65f4c-4337-4e7a-95c5-be6c92783350 service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:06.353749 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 919/1870] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:26:06 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:26:06.789566 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024ea3b4-5577-4c5c-97a5-1ed83f31a8f8 service cinder] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:06.799253 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024ea3b4-5577-4c5c-97a5-1ed83f31a8f8 service cinder] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:06.813880 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be8ed7c-47b6-42af-ae39-c5e32709d4f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e31f2728056b460fa474478eb85ca1d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:06.814530 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be8ed7c-47b6-42af-ae39-c5e32709d4f9 service cinder] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:06.814814 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be8ed7c-47b6-42af-ae39-c5e32709d4f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:06.815956 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be8ed7c-47b6-42af-ae39-c5e32709d4f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:06.896527 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be8ed7c-47b6-42af-ae39-c5e32709d4f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:06.898934 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be8ed7c-47b6-42af-ae39-c5e32709d4f9 service cinder] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:06.925162 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 952/1871] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:26:06 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:26:07.101081 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce65f4c-4337-4e7a-95c5-be6c92783350 service cinder] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:07.251670 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce65f4c-4337-4e7a-95c5-be6c92783350 service cinder] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:07.261087 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f3e9d7-31e5-4a49-b69d-6a24ebaef2a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:07.262279 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f3e9d7-31e5-4a49-b69d-6a24ebaef2a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:07.262883 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f3e9d7-31e5-4a49-b69d-6a24ebaef2a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:07.263900 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f3e9d7-31e5-4a49-b69d-6a24ebaef2a9 admin admin] PATH_INFO: `/v3/users/32ed30b6d278486ab1cd2eb6e352e344` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:07.265957 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f3e9d7-31e5-4a49-b69d-6a24ebaef2a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32ed30b6d278486ab1cd2eb6e352e344)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:07.270625 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f3e9d7-31e5-4a49-b69d-6a24ebaef2a9 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:07.542302 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-11f3e9d7-31e5-4a49-b69d-6a24ebaef2a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32ed30b6d278486ab1cd2eb6e352e344'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:07.560184 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-11f3e9d7-31e5-4a49-b69d-6a24ebaef2a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32ed30b6d278486ab1cd2eb6e352e344'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:07.561107 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-11f3e9d7-31e5-4a49-b69d-6a24ebaef2a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32ed30b6d278486ab1cd2eb6e352e344'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:07.571833 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-11f3e9d7-31e5-4a49-b69d-6a24ebaef2a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32ed30b6d278486ab1cd2eb6e352e344'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:07.572519 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-11f3e9d7-31e5-4a49-b69d-6a24ebaef2a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32ed30b6d278486ab1cd2eb6e352e344'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:07.729723 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 920/1872] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:26:07 2022] DELETE /identity/v3/users/32ed30b6d278486ab1cd2eb6e352e344 => generated 0 bytes in 626 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:26:07.730584 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedea707-7738-4502-b903-79980b1a0800 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:07.731080 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedea707-7738-4502-b903-79980b1a0800 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:07.731186 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedea707-7738-4502-b903-79980b1a0800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:07.748095 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:26:07.754309 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-eedea707-7738-4502-b903-79980b1a0800 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:26:07.755634 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-eedea707-7738-4502-b903-79980b1a0800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:07.856987 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 953/1873] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:26:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:26:07.870605 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f3e9d7-31e5-4a49-b69d-6a24ebaef2a9 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:07.872354 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-11f3e9d7-31e5-4a49-b69d-6a24ebaef2a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:07.937497 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f3e9d7-31e5-4a49-b69d-6a24ebaef2a9 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:07.942321 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a3f9ca-2454-44ff-adcb-fdd6680cd8fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:07.943061 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a3f9ca-2454-44ff-adcb-fdd6680cd8fe service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:07.943515 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a3f9ca-2454-44ff-adcb-fdd6680cd8fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:07.943892 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a3f9ca-2454-44ff-adcb-fdd6680cd8fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:07.955805 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a3f9ca-2454-44ff-adcb-fdd6680cd8fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:07.960418 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a3f9ca-2454-44ff-adcb-fdd6680cd8fe service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:07.970229 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 921/1874] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:26:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:26:08.456855 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:26:08.537246 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedea707-7738-4502-b903-79980b1a0800 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:08.537246 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-eedea707-7738-4502-b903-79980b1a0800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:08.559230 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedea707-7738-4502-b903-79980b1a0800 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:08.575592 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cbd37a-2760-48a3-b88f-d3b2eea91efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:08.577608 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cbd37a-2760-48a3-b88f-d3b2eea91efe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:08.578427 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cbd37a-2760-48a3-b88f-d3b2eea91efe admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:08.579992 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cbd37a-2760-48a3-b88f-d3b2eea91efe admin admin] PATH_INFO: `/v3/projects/b51a1b54a7554b408dfaaf90e0337156` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:08.581550 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cbd37a-2760-48a3-b88f-d3b2eea91efe admin admin] RBAC: Authorizing `identity:delete_project(project_id=b51a1b54a7554b408dfaaf90e0337156)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:08.583592 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cbd37a-2760-48a3-b88f-d3b2eea91efe admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:08.764612 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-d0cbd37a-2760-48a3-b88f-d3b2eea91efe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b51a1b54a7554b408dfaaf90e0337156', 'request_id': 'req-d0cbd37a-2760-48a3-b88f-d3b2eea91efe'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:08.770078 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-d0cbd37a-2760-48a3-b88f-d3b2eea91efe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b51a1b54a7554b408dfaaf90e0337156', 'request_id': 'req-d0cbd37a-2760-48a3-b88f-d3b2eea91efe'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:08.878527 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-d0cbd37a-2760-48a3-b88f-d3b2eea91efe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b51a1b54a7554b408dfaaf90e0337156', 'request_id': 'req-d0cbd37a-2760-48a3-b88f-d3b2eea91efe'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:08.891594 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-d0cbd37a-2760-48a3-b88f-d3b2eea91efe admin admin] The token cache is being invalidate because project b51a1b54a7554b408dfaaf90e0337156 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:26:08.891890 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-d0cbd37a-2760-48a3-b88f-d3b2eea91efe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:08.892586 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-d0cbd37a-2760-48a3-b88f-d3b2eea91efe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:08.894413 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 954/1875] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:26:08 2022] DELETE /identity/v3/projects/b51a1b54a7554b408dfaaf90e0337156 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:26:08.898586 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a3f9ca-2454-44ff-adcb-fdd6680cd8fe service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:08.912849 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a3f9ca-2454-44ff-adcb-fdd6680cd8fe service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:08.987104 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2159f6a-86ac-49f7-95bb-789b2d3d51e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:08.987789 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2159f6a-86ac-49f7-95bb-789b2d3d51e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:08.988072 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2159f6a-86ac-49f7-95bb-789b2d3d51e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:08.992753 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2159f6a-86ac-49f7-95bb-789b2d3d51e4 admin admin] PATH_INFO: `/v3/users/ea0efe59c2974606bb77e6cd0882f113` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:08.992753 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2159f6a-86ac-49f7-95bb-789b2d3d51e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea0efe59c2974606bb77e6cd0882f113)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:08.992753 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2159f6a-86ac-49f7-95bb-789b2d3d51e4 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:09.241585 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-f2159f6a-86ac-49f7-95bb-789b2d3d51e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea0efe59c2974606bb77e6cd0882f113'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:09.267296 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-f2159f6a-86ac-49f7-95bb-789b2d3d51e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea0efe59c2974606bb77e6cd0882f113'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:09.268569 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-f2159f6a-86ac-49f7-95bb-789b2d3d51e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea0efe59c2974606bb77e6cd0882f113'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:09.277930 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-f2159f6a-86ac-49f7-95bb-789b2d3d51e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea0efe59c2974606bb77e6cd0882f113'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:09.278952 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-f2159f6a-86ac-49f7-95bb-789b2d3d51e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea0efe59c2974606bb77e6cd0882f113'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:09.388903 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:26:09.411179 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 922/1876] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:26:08 2022] DELETE /identity/v3/users/ea0efe59c2974606bb77e6cd0882f113 => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:26:10.128845 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cbd37a-2760-48a3-b88f-d3b2eea91efe admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:10.130039 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-d0cbd37a-2760-48a3-b88f-d3b2eea91efe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:10.151813 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cbd37a-2760-48a3-b88f-d3b2eea91efe admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:10.240813 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bec14aa-7bdf-4265-b60d-135332c705c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:10.242057 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bec14aa-7bdf-4265-b60d-135332c705c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:10.242509 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bec14aa-7bdf-4265-b60d-135332c705c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:10.243035 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bec14aa-7bdf-4265-b60d-135332c705c4 admin admin] PATH_INFO: `/v3/projects/0006aad3f9d14cc896663a5c5a558dd0` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:10.244504 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bec14aa-7bdf-4265-b60d-135332c705c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0006aad3f9d14cc896663a5c5a558dd0)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:10.247347 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bec14aa-7bdf-4265-b60d-135332c705c4 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:10.268042 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:26:10.374130 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-4bec14aa-7bdf-4265-b60d-135332c705c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0006aad3f9d14cc896663a5c5a558dd0', 'request_id': 'req-4bec14aa-7bdf-4265-b60d-135332c705c4'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:10.383064 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-4bec14aa-7bdf-4265-b60d-135332c705c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0006aad3f9d14cc896663a5c5a558dd0', 'request_id': 'req-4bec14aa-7bdf-4265-b60d-135332c705c4'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:10.507522 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-4bec14aa-7bdf-4265-b60d-135332c705c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0006aad3f9d14cc896663a5c5a558dd0', 'request_id': 'req-4bec14aa-7bdf-4265-b60d-135332c705c4'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:10.533287 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-4bec14aa-7bdf-4265-b60d-135332c705c4 admin admin] The token cache is being invalidate because project 0006aad3f9d14cc896663a5c5a558dd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:26:10.533287 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-4bec14aa-7bdf-4265-b60d-135332c705c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:10.554181 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-4bec14aa-7bdf-4265-b60d-135332c705c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:10.558764 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 955/1877] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:26:10 2022] DELETE /identity/v3/projects/0006aad3f9d14cc896663a5c5a558dd0 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:26:10.565146 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be693464-aecf-494c-beb1-dcb51e37354d None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:10.565913 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be693464-aecf-494c-beb1-dcb51e37354d None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:10.565913 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be693464-aecf-494c-beb1-dcb51e37354d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:10.585654 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-be693464-aecf-494c-beb1-dcb51e37354d None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:26:10.587118 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-be693464-aecf-494c-beb1-dcb51e37354d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:10.659180 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:26:10.706752 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 923/1878] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:26:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:26:10.709260 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280880f7-1e03-4679-a56b-6b81e925e934 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:10.709560 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280880f7-1e03-4679-a56b-6b81e925e934 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:10.709835 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280880f7-1e03-4679-a56b-6b81e925e934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:10.763000 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-280880f7-1e03-4679-a56b-6b81e925e934 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:26:10.765570 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-280880f7-1e03-4679-a56b-6b81e925e934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:10.785302 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 956/1879] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:26:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:26:10.790116 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be693464-aecf-494c-beb1-dcb51e37354d None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:10.809311 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be693464-aecf-494c-beb1-dcb51e37354d None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:10.819396 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68532abe-01a7-4330-b523-bfc4fb3d8f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:10.820715 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68532abe-01a7-4330-b523-bfc4fb3d8f7a admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:10.821435 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68532abe-01a7-4330-b523-bfc4fb3d8f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:10.822143 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68532abe-01a7-4330-b523-bfc4fb3d8f7a admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:10.824477 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68532abe-01a7-4330-b523-bfc4fb3d8f7a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:26:10.825301 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68532abe-01a7-4330-b523-bfc4fb3d8f7a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:10.831335 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68532abe-01a7-4330-b523-bfc4fb3d8f7a admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:10.861145 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 924/1880] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Aug 4 21:26:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:26:10.867187 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280880f7-1e03-4679-a56b-6b81e925e934 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:10.897947 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280880f7-1e03-4679-a56b-6b81e925e934 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:10.928483 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d16ff9-142c-401c-a314-791191765b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:10.928483 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d16ff9-142c-401c-a314-791191765b53 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:10.928483 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d16ff9-142c-401c-a314-791191765b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:10.928483 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d16ff9-142c-401c-a314-791191765b53 admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:10.928758 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d16ff9-142c-401c-a314-791191765b53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:10.928758 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d16ff9-142c-401c-a314-791191765b53 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:11.077253 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 957/1881] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:26:10 2022] POST /identity/v3/projects => generated 377 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:26:11.077382 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68532abe-01a7-4330-b523-bfc4fb3d8f7a admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:11.091650 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68532abe-01a7-4330-b523-bfc4fb3d8f7a admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:11.240579 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d16ff9-142c-401c-a314-791191765b53 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:11.243972 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-34d16ff9-142c-401c-a314-791191765b53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:11.256439 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2589570-7904-49d3-9d42-58431db0d854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:11.258976 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2589570-7904-49d3-9d42-58431db0d854 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:11.260059 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2589570-7904-49d3-9d42-58431db0d854 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:11.264452 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2589570-7904-49d3-9d42-58431db0d854 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:11.278315 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2589570-7904-49d3-9d42-58431db0d854 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:11.278509 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d16ff9-142c-401c-a314-791191765b53 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:11.278655 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2589570-7904-49d3-9d42-58431db0d854 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:11.291601 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd2c6b3-b14d-4841-94bc-ccbe82a3af86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:11.292375 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd2c6b3-b14d-4841-94bc-ccbe82a3af86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:11.292739 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd2c6b3-b14d-4841-94bc-ccbe82a3af86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:11.293167 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd2c6b3-b14d-4841-94bc-ccbe82a3af86 admin admin] PATH_INFO: `/v3/users/4f0830ef216244f0a73a8f164ad26a31` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:11.295045 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd2c6b3-b14d-4841-94bc-ccbe82a3af86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f0830ef216244f0a73a8f164ad26a31)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:11.298989 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd2c6b3-b14d-4841-94bc-ccbe82a3af86 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:11.414322 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 925/1882] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:26:11 2022] POST /identity/v3/users => generated 289 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:26:11.419626 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2589570-7904-49d3-9d42-58431db0d854 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:11.435719 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2589570-7904-49d3-9d42-58431db0d854 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:11.444054 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c14cc8f-f41f-4919-a0c2-8b19397c633c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:11.444963 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c14cc8f-f41f-4919-a0c2-8b19397c633c admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:11.445452 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c14cc8f-f41f-4919-a0c2-8b19397c633c admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:11.445863 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c14cc8f-f41f-4919-a0c2-8b19397c633c admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:11.447940 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c14cc8f-f41f-4919-a0c2-8b19397c633c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:11.450479 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c14cc8f-f41f-4919-a0c2-8b19397c633c admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:11.476918 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 926/1883] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:26:11 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:26:11.486808 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c14cc8f-f41f-4919-a0c2-8b19397c633c admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:11.500916 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c14cc8f-f41f-4919-a0c2-8b19397c633c admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:11.506736 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f38a2f0-d75e-4144-a8f6-f30fd09c377a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:11.507532 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f38a2f0-d75e-4144-a8f6-f30fd09c377a admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:11.508094 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f38a2f0-d75e-4144-a8f6-f30fd09c377a admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:11.508475 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f38a2f0-d75e-4144-a8f6-f30fd09c377a admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:11.509290 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f38a2f0-d75e-4144-a8f6-f30fd09c377a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:11.511047 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f38a2f0-d75e-4144-a8f6-f30fd09c377a admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:11.523549 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 927/1884] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:26:11 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:26:11.529209 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f38a2f0-d75e-4144-a8f6-f30fd09c377a admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:11.543741 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f38a2f0-d75e-4144-a8f6-f30fd09c377a admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:11.549831 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8983707c-a607-4081-8a18-018d057f1f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:11.550543 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8983707c-a607-4081-8a18-018d057f1f71 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:11.550930 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8983707c-a607-4081-8a18-018d057f1f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:11.551432 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8983707c-a607-4081-8a18-018d057f1f71 admin admin] PATH_INFO: `/v3/projects/53d151c2cb8f49be8adbbe0dece8e60b/users/4412c61b45974da4acbb35fb35fe449c/roles/8feebc60363c4feb870718b1721e5324` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:11.553487 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-dbd2c6b3-b14d-4841-94bc-ccbe82a3af86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f0830ef216244f0a73a8f164ad26a31'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:11.570428 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8983707c-a607-4081-8a18-018d057f1f71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53d151c2cb8f49be8adbbe0dece8e60b, user_id=4412c61b45974da4acbb35fb35fe449c, role_id=8feebc60363c4feb870718b1721e5324)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:11.578390 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8983707c-a607-4081-8a18-018d057f1f71 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:11.579163 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-dbd2c6b3-b14d-4841-94bc-ccbe82a3af86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f0830ef216244f0a73a8f164ad26a31'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:11.580740 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-dbd2c6b3-b14d-4841-94bc-ccbe82a3af86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f0830ef216244f0a73a8f164ad26a31'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:11.581713 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-dbd2c6b3-b14d-4841-94bc-ccbe82a3af86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f0830ef216244f0a73a8f164ad26a31'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:11.668398 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 928/1885] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:26:11 2022] PUT /identity/v3/projects/53d151c2cb8f49be8adbbe0dece8e60b/users/4412c61b45974da4acbb35fb35fe449c/roles/8feebc60363c4feb870718b1721e5324 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:26:11.673032 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-dbd2c6b3-b14d-4841-94bc-ccbe82a3af86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f0830ef216244f0a73a8f164ad26a31'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:11.689489 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb317d95-c16f-4e1e-bbfb-b0da276db27c None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:11.690413 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb317d95-c16f-4e1e-bbfb-b0da276db27c None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:11.691598 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb317d95-c16f-4e1e-bbfb-b0da276db27c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:11.700482 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 958/1886] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:26:11 2022] DELETE /identity/v3/users/4f0830ef216244f0a73a8f164ad26a31 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:26:11.706718 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa7ea37-3708-4682-a07a-c8cfea8f1373 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:11.707277 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa7ea37-3708-4682-a07a-c8cfea8f1373 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:11.707744 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa7ea37-3708-4682-a07a-c8cfea8f1373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:11.734284 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-6fa7ea37-3708-4682-a07a-c8cfea8f1373 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:26:11.736869 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-6fa7ea37-3708-4682-a07a-c8cfea8f1373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:11.767132 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-bb317d95-c16f-4e1e-bbfb-b0da276db27c None None] MFA Rules not processed for user `4412c61b45974da4acbb35fb35fe449c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:26:11.771276 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-bb317d95-c16f-4e1e-bbfb-b0da276db27c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:11.831625 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:26:11.880296 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 959/1887] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:26:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:26:11.895634 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa7ea37-3708-4682-a07a-c8cfea8f1373 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:11.898447 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-6fa7ea37-3708-4682-a07a-c8cfea8f1373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:11.921899 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 929/1888] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:26:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:26:11.929209 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb317d95-c16f-4e1e-bbfb-b0da276db27c None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:11.932116 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-bb317d95-c16f-4e1e-bbfb-b0da276db27c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:11.967162 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb317d95-c16f-4e1e-bbfb-b0da276db27c None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:11.982705 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f47751-c1f5-440b-a5c0-314cf85f1772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:11.984030 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f47751-c1f5-440b-a5c0-314cf85f1772 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:11.984710 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f47751-c1f5-440b-a5c0-314cf85f1772 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:11.985271 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f47751-c1f5-440b-a5c0-314cf85f1772 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:11.987351 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f47751-c1f5-440b-a5c0-314cf85f1772 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:11.992421 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f47751-c1f5-440b-a5c0-314cf85f1772 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:12.013802 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:26:12.048082 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa7ea37-3708-4682-a07a-c8cfea8f1373 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:12.052256 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a053132-aa5e-4beb-9077-1358167b63e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:12.053166 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a053132-aa5e-4beb-9077-1358167b63e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:12.053521 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a053132-aa5e-4beb-9077-1358167b63e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:12.054238 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a053132-aa5e-4beb-9077-1358167b63e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:12.067416 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a053132-aa5e-4beb-9077-1358167b63e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:12.069024 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a053132-aa5e-4beb-9077-1358167b63e1 service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:12.086892 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 960/1889] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:26:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:26:12.149894 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a053132-aa5e-4beb-9077-1358167b63e1 service neutron] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:12.155958 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 930/1890] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:26:11 2022] POST /identity/v3/projects => generated 379 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:26:12.169628 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a053132-aa5e-4beb-9077-1358167b63e1 service neutron] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:12.334490 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ede752-5e20-4909-b36e-b5a10e452964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:12.335992 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ede752-5e20-4909-b36e-b5a10e452964 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:12.336421 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ede752-5e20-4909-b36e-b5a10e452964 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:12.336868 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ede752-5e20-4909-b36e-b5a10e452964 admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:12.338077 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ede752-5e20-4909-b36e-b5a10e452964 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:12.340484 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ede752-5e20-4909-b36e-b5a10e452964 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:12.461876 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 961/1891] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:26:12 2022] POST /identity/v3/users => generated 296 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:26:12.465695 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f47751-c1f5-440b-a5c0-314cf85f1772 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:12.491961 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f47751-c1f5-440b-a5c0-314cf85f1772 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:12.503789 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c890dd14-f99d-49cb-92da-a9e6e52c48e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:12.504549 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c890dd14-f99d-49cb-92da-a9e6e52c48e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:12.504994 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c890dd14-f99d-49cb-92da-a9e6e52c48e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:12.505468 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c890dd14-f99d-49cb-92da-a9e6e52c48e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:12.511409 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c890dd14-f99d-49cb-92da-a9e6e52c48e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:12.514404 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c890dd14-f99d-49cb-92da-a9e6e52c48e7 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:12.529669 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 931/1892] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:26:12 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:26:12.533871 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ede752-5e20-4909-b36e-b5a10e452964 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:12.544895 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ede752-5e20-4909-b36e-b5a10e452964 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:12.560393 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb17e173-9300-4ded-9895-30f4ff290893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:12.560393 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb17e173-9300-4ded-9895-30f4ff290893 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:12.560393 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb17e173-9300-4ded-9895-30f4ff290893 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:12.560393 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb17e173-9300-4ded-9895-30f4ff290893 admin admin] PATH_INFO: `/v3/projects/71bf4508e4d64f87921b60b2fdced501/users/a6ab851f3a5c470d993bcdecd5db89cb/roles/a432dd4735944ecfab2fa5ce1705d356` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:12.581345 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb17e173-9300-4ded-9895-30f4ff290893 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71bf4508e4d64f87921b60b2fdced501, user_id=a6ab851f3a5c470d993bcdecd5db89cb, role_id=a432dd4735944ecfab2fa5ce1705d356)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:12.586060 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb17e173-9300-4ded-9895-30f4ff290893 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:12.736277 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 962/1893] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:26:12 2022] PUT /identity/v3/projects/71bf4508e4d64f87921b60b2fdced501/users/a6ab851f3a5c470d993bcdecd5db89cb/roles/a432dd4735944ecfab2fa5ce1705d356 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:26:12.740689 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c890dd14-f99d-49cb-92da-a9e6e52c48e7 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:12.754608 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c890dd14-f99d-49cb-92da-a9e6e52c48e7 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:12.829210 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cff18db-dec3-4ce9-a489-21febefca82e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:12.830247 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cff18db-dec3-4ce9-a489-21febefca82e admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:12.831802 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cff18db-dec3-4ce9-a489-21febefca82e admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:12.832426 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cff18db-dec3-4ce9-a489-21febefca82e admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:12.833997 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cff18db-dec3-4ce9-a489-21febefca82e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:12.837272 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cff18db-dec3-4ce9-a489-21febefca82e admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:12.851077 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 932/1894] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:26:12 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:26:12.855365 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb17e173-9300-4ded-9895-30f4ff290893 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:12.880794 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb17e173-9300-4ded-9895-30f4ff290893 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:12.888153 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8218e50c-f1e6-458a-807c-f58d0b2659a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:12.889026 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8218e50c-f1e6-458a-807c-f58d0b2659a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:12.889311 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8218e50c-f1e6-458a-807c-f58d0b2659a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:12.889791 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8218e50c-f1e6-458a-807c-f58d0b2659a2 admin admin] PATH_INFO: `/v3/projects/71bf4508e4d64f87921b60b2fdced501/users/a6ab851f3a5c470d993bcdecd5db89cb/roles/a432dd4735944ecfab2fa5ce1705d356` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:12.891933 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8218e50c-f1e6-458a-807c-f58d0b2659a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71bf4508e4d64f87921b60b2fdced501, user_id=a6ab851f3a5c470d993bcdecd5db89cb, role_id=a432dd4735944ecfab2fa5ce1705d356)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:12.894876 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8218e50c-f1e6-458a-807c-f58d0b2659a2 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:12.908577 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 963/1895] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:26:12 2022] PUT /identity/v3/projects/71bf4508e4d64f87921b60b2fdced501/users/a6ab851f3a5c470d993bcdecd5db89cb/roles/a432dd4735944ecfab2fa5ce1705d356 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:26:12.925651 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f036685f-a3ff-4832-b935-4dcb76214996 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:12.925996 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f036685f-a3ff-4832-b935-4dcb76214996 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:12.926215 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f036685f-a3ff-4832-b935-4dcb76214996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:12.968572 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-f036685f-a3ff-4832-b935-4dcb76214996 None None] MFA Rules not processed for user `a6ab851f3a5c470d993bcdecd5db89cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:26:12.970203 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-f036685f-a3ff-4832-b935-4dcb76214996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:13.154491 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:26:13.166981 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 933/1896] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:26:12 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:26:13.203736 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8218e50c-f1e6-458a-807c-f58d0b2659a2 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:13.205970 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-8218e50c-f1e6-458a-807c-f58d0b2659a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:13.290371 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:26:13.377714 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8218e50c-f1e6-458a-807c-f58d0b2659a2 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:13.383191 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3c0e2c-dc68-4bbd-a765-54c3b68b707a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e31f2728056b460fa474478eb85ca1d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:13.384247 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3c0e2c-dc68-4bbd-a765-54c3b68b707a service cinder] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:13.384532 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3c0e2c-dc68-4bbd-a765-54c3b68b707a service cinder] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:13.385628 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3c0e2c-dc68-4bbd-a765-54c3b68b707a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:13.396527 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3c0e2c-dc68-4bbd-a765-54c3b68b707a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:13.399072 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3c0e2c-dc68-4bbd-a765-54c3b68b707a service cinder] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:13.414755 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 964/1897] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:26:13 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:26:13.926490 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f036685f-a3ff-4832-b935-4dcb76214996 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:13.929233 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-f036685f-a3ff-4832-b935-4dcb76214996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:14.133687 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f036685f-a3ff-4832-b935-4dcb76214996 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:14.148091 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9113b5db-50ae-4fb7-99b0-4d75b5dcb6c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:14.148925 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9113b5db-50ae-4fb7-99b0-4d75b5dcb6c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:14.149276 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9113b5db-50ae-4fb7-99b0-4d75b5dcb6c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:14.150337 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9113b5db-50ae-4fb7-99b0-4d75b5dcb6c6 admin admin] PATH_INFO: `/v3/projects/48c6cf8ce81d4fe59668a421906b751c` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:14.151830 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9113b5db-50ae-4fb7-99b0-4d75b5dcb6c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48c6cf8ce81d4fe59668a421906b751c)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:14.154202 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9113b5db-50ae-4fb7-99b0-4d75b5dcb6c6 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:14.276378 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3c0e2c-dc68-4bbd-a765-54c3b68b707a service cinder] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:14.290926 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3c0e2c-dc68-4bbd-a765-54c3b68b707a service cinder] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:14.304676 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0321bb8d-31a1-4086-b94b-fee2b51727ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:14.305348 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0321bb8d-31a1-4086-b94b-fee2b51727ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:14.305689 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0321bb8d-31a1-4086-b94b-fee2b51727ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:14.305989 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0321bb8d-31a1-4086-b94b-fee2b51727ba admin admin] PATH_INFO: `/v3/users/4412c61b45974da4acbb35fb35fe449c` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:14.307410 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0321bb8d-31a1-4086-b94b-fee2b51727ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=4412c61b45974da4acbb35fb35fe449c)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:14.310202 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0321bb8d-31a1-4086-b94b-fee2b51727ba admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:14.333165 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-9113b5db-50ae-4fb7-99b0-4d75b5dcb6c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48c6cf8ce81d4fe59668a421906b751c', 'request_id': 'req-9113b5db-50ae-4fb7-99b0-4d75b5dcb6c6'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:14.346254 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-9113b5db-50ae-4fb7-99b0-4d75b5dcb6c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48c6cf8ce81d4fe59668a421906b751c', 'request_id': 'req-9113b5db-50ae-4fb7-99b0-4d75b5dcb6c6'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:14.496969 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-9113b5db-50ae-4fb7-99b0-4d75b5dcb6c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48c6cf8ce81d4fe59668a421906b751c', 'request_id': 'req-9113b5db-50ae-4fb7-99b0-4d75b5dcb6c6'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:14.516244 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-9113b5db-50ae-4fb7-99b0-4d75b5dcb6c6 admin admin] The token cache is being invalidate because project 48c6cf8ce81d4fe59668a421906b751c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:26:14.517927 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-9113b5db-50ae-4fb7-99b0-4d75b5dcb6c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:14.519220 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-9113b5db-50ae-4fb7-99b0-4d75b5dcb6c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:14.554419 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 934/1898] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:26:13 2022] DELETE /identity/v3/projects/48c6cf8ce81d4fe59668a421906b751c => generated 0 bytes in 600 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:26:14.554419 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9113b5db-50ae-4fb7-99b0-4d75b5dcb6c6 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:14.554419 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-9113b5db-50ae-4fb7-99b0-4d75b5dcb6c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:14.648513 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9113b5db-50ae-4fb7-99b0-4d75b5dcb6c6 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:14.652843 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66930dd3-943c-4fbe-a458-a099d7775653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:14.653889 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66930dd3-943c-4fbe-a458-a099d7775653 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:14.654256 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66930dd3-943c-4fbe-a458-a099d7775653 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:14.654643 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66930dd3-943c-4fbe-a458-a099d7775653 admin admin] PATH_INFO: `/v3/users/b622ea9bfe0546ddb17033ada9ce255a` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:14.655984 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66930dd3-943c-4fbe-a458-a099d7775653 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b622ea9bfe0546ddb17033ada9ce255a)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:14.658558 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66930dd3-943c-4fbe-a458-a099d7775653 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:14.744376 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:26:14.812080 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-0321bb8d-31a1-4086-b94b-fee2b51727ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4412c61b45974da4acbb35fb35fe449c'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:14.831560 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-0321bb8d-31a1-4086-b94b-fee2b51727ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4412c61b45974da4acbb35fb35fe449c'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:14.833197 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-0321bb8d-31a1-4086-b94b-fee2b51727ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4412c61b45974da4acbb35fb35fe449c'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:14.834143 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-0321bb8d-31a1-4086-b94b-fee2b51727ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4412c61b45974da4acbb35fb35fe449c'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:14.923613 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-0321bb8d-31a1-4086-b94b-fee2b51727ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4412c61b45974da4acbb35fb35fe449c'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:14.947420 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 965/1899] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:26:14 2022] DELETE /identity/v3/users/4412c61b45974da4acbb35fb35fe449c => generated 0 bytes in 671 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:26:14.952966 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79065608-b0cd-4380-9a03-f3a480304130 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:14.953359 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79065608-b0cd-4380-9a03-f3a480304130 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:14.953742 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79065608-b0cd-4380-9a03-f3a480304130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:15.015005 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-79065608-b0cd-4380-9a03-f3a480304130 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:26:15.016298 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-79065608-b0cd-4380-9a03-f3a480304130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:15.070014 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-66930dd3-943c-4fbe-a458-a099d7775653 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b622ea9bfe0546ddb17033ada9ce255a'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:15.103060 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-66930dd3-943c-4fbe-a458-a099d7775653 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b622ea9bfe0546ddb17033ada9ce255a'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:15.104549 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-66930dd3-943c-4fbe-a458-a099d7775653 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b622ea9bfe0546ddb17033ada9ce255a'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:15.118844 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:26:15.124464 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-66930dd3-943c-4fbe-a458-a099d7775653 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b622ea9bfe0546ddb17033ada9ce255a'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:15.126862 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-66930dd3-943c-4fbe-a458-a099d7775653 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b622ea9bfe0546ddb17033ada9ce255a'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:15.178687 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 966/1900] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:26:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:26:15.217444 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79065608-b0cd-4380-9a03-f3a480304130 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:15.223308 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-79065608-b0cd-4380-9a03-f3a480304130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:15.246563 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 935/1901] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:26:14 2022] DELETE /identity/v3/users/b622ea9bfe0546ddb17033ada9ce255a => generated 0 bytes in 719 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:26:15.379631 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79065608-b0cd-4380-9a03-f3a480304130 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:15.471398 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5adc43-6e29-4ced-969b-1c034aacff9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:15.473733 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5adc43-6e29-4ced-969b-1c034aacff9f service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:15.474414 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5adc43-6e29-4ced-969b-1c034aacff9f service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:15.475091 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5adc43-6e29-4ced-969b-1c034aacff9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:15.598303 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5adc43-6e29-4ced-969b-1c034aacff9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:15.600461 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5adc43-6e29-4ced-969b-1c034aacff9f service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:15.617088 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 967/1902] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:26:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:26:16.135119 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66930dd3-943c-4fbe-a458-a099d7775653 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:16.139510 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-66930dd3-943c-4fbe-a458-a099d7775653 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:16.166471 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66930dd3-943c-4fbe-a458-a099d7775653 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:16.172406 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa5bcb3-b4f5-491b-a8b0-d27c98e0bfd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:16.173361 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa5bcb3-b4f5-491b-a8b0-d27c98e0bfd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:16.173821 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa5bcb3-b4f5-491b-a8b0-d27c98e0bfd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:16.174222 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa5bcb3-b4f5-491b-a8b0-d27c98e0bfd9 admin admin] PATH_INFO: `/v3/projects/3c7268e7eb3d4a71bcca41919a3b655f` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:16.175876 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa5bcb3-b4f5-491b-a8b0-d27c98e0bfd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c7268e7eb3d4a71bcca41919a3b655f)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:16.178914 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa5bcb3-b4f5-491b-a8b0-d27c98e0bfd9 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:16.242577 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5adc43-6e29-4ced-969b-1c034aacff9f service neutron] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:16.248512 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-ba5adc43-6e29-4ced-969b-1c034aacff9f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:16.279551 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5adc43-6e29-4ced-969b-1c034aacff9f service neutron] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:16.286135 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cabc11-cde6-461d-87a4-2a241af93e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:16.287368 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cabc11-cde6-461d-87a4-2a241af93e4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:16.287807 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cabc11-cde6-461d-87a4-2a241af93e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:16.288252 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cabc11-cde6-461d-87a4-2a241af93e4d admin admin] PATH_INFO: `/v3/projects/53d151c2cb8f49be8adbbe0dece8e60b` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:16.290898 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cabc11-cde6-461d-87a4-2a241af93e4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=53d151c2cb8f49be8adbbe0dece8e60b)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:16.295129 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cabc11-cde6-461d-87a4-2a241af93e4d admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:16.384287 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-8aa5bcb3-b4f5-491b-a8b0-d27c98e0bfd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c7268e7eb3d4a71bcca41919a3b655f', 'request_id': 'req-8aa5bcb3-b4f5-491b-a8b0-d27c98e0bfd9'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:16.392135 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-8aa5bcb3-b4f5-491b-a8b0-d27c98e0bfd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c7268e7eb3d4a71bcca41919a3b655f', 'request_id': 'req-8aa5bcb3-b4f5-491b-a8b0-d27c98e0bfd9'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:16.435352 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-d6cabc11-cde6-461d-87a4-2a241af93e4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53d151c2cb8f49be8adbbe0dece8e60b', 'request_id': 'req-d6cabc11-cde6-461d-87a4-2a241af93e4d'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:16.444803 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-d6cabc11-cde6-461d-87a4-2a241af93e4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53d151c2cb8f49be8adbbe0dece8e60b', 'request_id': 'req-d6cabc11-cde6-461d-87a4-2a241af93e4d'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:16.472986 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-8aa5bcb3-b4f5-491b-a8b0-d27c98e0bfd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c7268e7eb3d4a71bcca41919a3b655f', 'request_id': 'req-8aa5bcb3-b4f5-491b-a8b0-d27c98e0bfd9'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:16.486431 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-8aa5bcb3-b4f5-491b-a8b0-d27c98e0bfd9 admin admin] The token cache is being invalidate because project 3c7268e7eb3d4a71bcca41919a3b655f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:26:16.486783 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-8aa5bcb3-b4f5-491b-a8b0-d27c98e0bfd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:16.487556 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-8aa5bcb3-b4f5-491b-a8b0-d27c98e0bfd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:16.489541 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 936/1903] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:26:16 2022] DELETE /identity/v3/projects/3c7268e7eb3d4a71bcca41919a3b655f => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:26:16.497205 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3930c207-5ad7-467e-94dd-9226ecec8dd5 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:16.497549 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3930c207-5ad7-467e-94dd-9226ecec8dd5 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:16.497831 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3930c207-5ad7-467e-94dd-9226ecec8dd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:16.519861 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-3930c207-5ad7-467e-94dd-9226ecec8dd5 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:26:16.521326 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-3930c207-5ad7-467e-94dd-9226ecec8dd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:16.567999 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-d6cabc11-cde6-461d-87a4-2a241af93e4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53d151c2cb8f49be8adbbe0dece8e60b', 'request_id': 'req-d6cabc11-cde6-461d-87a4-2a241af93e4d'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:16.595373 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-d6cabc11-cde6-461d-87a4-2a241af93e4d admin admin] The token cache is being invalidate because project 53d151c2cb8f49be8adbbe0dece8e60b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:26:16.595373 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-d6cabc11-cde6-461d-87a4-2a241af93e4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:16.596390 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-d6cabc11-cde6-461d-87a4-2a241af93e4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:16.598314 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 968/1904] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:26:16 2022] DELETE /identity/v3/projects/53d151c2cb8f49be8adbbe0dece8e60b => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:26:16.602933 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cabc11-cde6-461d-87a4-2a241af93e4d admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:16.605273 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-d6cabc11-cde6-461d-87a4-2a241af93e4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:16.623294 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 937/1905] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:26:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:26:16.633769 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb61876-669d-4826-bed8-392b6fbd2797 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:16.633769 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb61876-669d-4826-bed8-392b6fbd2797 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:16.633769 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb61876-669d-4826-bed8-392b6fbd2797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:16.664347 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-edb61876-669d-4826-bed8-392b6fbd2797 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:26:16.666180 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-edb61876-669d-4826-bed8-392b6fbd2797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:16.679722 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 938/1906] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:26:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:26:16.684827 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb61876-669d-4826-bed8-392b6fbd2797 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:16.698207 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb61876-669d-4826-bed8-392b6fbd2797 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:16.703810 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e008651-d834-4b18-b20c-d4ea289bbd30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:16.705157 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e008651-d834-4b18-b20c-d4ea289bbd30 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:16.705157 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e008651-d834-4b18-b20c-d4ea289bbd30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:16.705157 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e008651-d834-4b18-b20c-d4ea289bbd30 admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:16.705851 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e008651-d834-4b18-b20c-d4ea289bbd30 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:26:16.706157 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e008651-d834-4b18-b20c-d4ea289bbd30 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:16.707849 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e008651-d834-4b18-b20c-d4ea289bbd30 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:16.709353 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cabc11-cde6-461d-87a4-2a241af93e4d admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:16.715142 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d3cb39-f854-48c8-b94f-7b2d7f8ec01a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:16.715850 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d3cb39-f854-48c8-b94f-7b2d7f8ec01a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:16.716133 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d3cb39-f854-48c8-b94f-7b2d7f8ec01a admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:16.716534 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d3cb39-f854-48c8-b94f-7b2d7f8ec01a admin admin] PATH_INFO: `/v3/users/a6ab851f3a5c470d993bcdecd5db89cb` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:16.717861 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d3cb39-f854-48c8-b94f-7b2d7f8ec01a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6ab851f3a5c470d993bcdecd5db89cb)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:16.737987 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d3cb39-f854-48c8-b94f-7b2d7f8ec01a admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:16.739855 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 939/1907] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Aug 4 21:26:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:26:16.739855 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e008651-d834-4b18-b20c-d4ea289bbd30 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:16.751283 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e008651-d834-4b18-b20c-d4ea289bbd30 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:16.758890 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83420563-9325-407b-82be-18f417994d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:16.759659 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83420563-9325-407b-82be-18f417994d57 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:16.759937 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83420563-9325-407b-82be-18f417994d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:16.760471 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83420563-9325-407b-82be-18f417994d57 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:16.761353 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83420563-9325-407b-82be-18f417994d57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:16.763424 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83420563-9325-407b-82be-18f417994d57 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:16.897029 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 940/1908] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:26:16 2022] POST /identity/v3/projects => generated 385 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:26:16.902068 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83420563-9325-407b-82be-18f417994d57 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:16.978638 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83420563-9325-407b-82be-18f417994d57 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:16.983145 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-06d3cb39-f854-48c8-b94f-7b2d7f8ec01a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6ab851f3a5c470d993bcdecd5db89cb'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:16.984416 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea566df8-39b0-4e27-8ab9-06d00e753de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:16.985778 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea566df8-39b0-4e27-8ab9-06d00e753de4 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:16.986059 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea566df8-39b0-4e27-8ab9-06d00e753de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:16.986964 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea566df8-39b0-4e27-8ab9-06d00e753de4 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:16.987814 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea566df8-39b0-4e27-8ab9-06d00e753de4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:16.989781 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea566df8-39b0-4e27-8ab9-06d00e753de4 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:16.997792 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-06d3cb39-f854-48c8-b94f-7b2d7f8ec01a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6ab851f3a5c470d993bcdecd5db89cb'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:16.998955 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-06d3cb39-f854-48c8-b94f-7b2d7f8ec01a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6ab851f3a5c470d993bcdecd5db89cb'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:16.999832 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-06d3cb39-f854-48c8-b94f-7b2d7f8ec01a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6ab851f3a5c470d993bcdecd5db89cb'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:17.090097 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:26:17.118809 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-06d3cb39-f854-48c8-b94f-7b2d7f8ec01a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6ab851f3a5c470d993bcdecd5db89cb'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:17.128063 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 941/1909] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:26:16 2022] POST /identity/v3/users => generated 293 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:26:17.132903 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea566df8-39b0-4e27-8ab9-06d00e753de4 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:17.137545 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-ea566df8-39b0-4e27-8ab9-06d00e753de4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:17.141192 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 969/1910] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:26:16 2022] DELETE /identity/v3/users/a6ab851f3a5c470d993bcdecd5db89cb => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:26:17.151565 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea566df8-39b0-4e27-8ab9-06d00e753de4 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:17.218465 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2ef27e-8924-41f0-bfbd-9624f9f385b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:17.219681 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2ef27e-8924-41f0-bfbd-9624f9f385b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:17.219971 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2ef27e-8924-41f0-bfbd-9624f9f385b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:17.220254 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2ef27e-8924-41f0-bfbd-9624f9f385b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:17.220968 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2ef27e-8924-41f0-bfbd-9624f9f385b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:17.222690 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2ef27e-8924-41f0-bfbd-9624f9f385b8 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:17.238467 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 942/1911] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:26:17 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:26:17.240371 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d3cb39-f854-48c8-b94f-7b2d7f8ec01a admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:17.251459 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d3cb39-f854-48c8-b94f-7b2d7f8ec01a admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:17.256551 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a733785-480b-4ee0-b2a6-d16ff22b9257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:17.257345 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a733785-480b-4ee0-b2a6-d16ff22b9257 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:17.257609 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a733785-480b-4ee0-b2a6-d16ff22b9257 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:17.257905 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a733785-480b-4ee0-b2a6-d16ff22b9257 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:17.258639 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a733785-480b-4ee0-b2a6-d16ff22b9257 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:17.260325 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a733785-480b-4ee0-b2a6-d16ff22b9257 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:17.274404 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 970/1912] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:26:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:26:17.277335 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2ef27e-8924-41f0-bfbd-9624f9f385b8 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:17.289709 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2ef27e-8924-41f0-bfbd-9624f9f385b8 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:17.294958 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f105c30b-15a2-49d3-8f57-4a62f1fe212c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:17.295718 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f105c30b-15a2-49d3-8f57-4a62f1fe212c admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:17.295990 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f105c30b-15a2-49d3-8f57-4a62f1fe212c admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:17.296419 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f105c30b-15a2-49d3-8f57-4a62f1fe212c admin admin] PATH_INFO: `/v3/projects/c4113b1df3374dca9515c8ec986688b1/users/b04345b7e0dd48d4a93a8be45c187464/roles/8feebc60363c4feb870718b1721e5324` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:17.313294 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f105c30b-15a2-49d3-8f57-4a62f1fe212c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4113b1df3374dca9515c8ec986688b1, user_id=b04345b7e0dd48d4a93a8be45c187464, role_id=8feebc60363c4feb870718b1721e5324)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:17.316624 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f105c30b-15a2-49d3-8f57-4a62f1fe212c admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:17.524718 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 943/1913] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:26:17 2022] PUT /identity/v3/projects/c4113b1df3374dca9515c8ec986688b1/users/b04345b7e0dd48d4a93a8be45c187464/roles/8feebc60363c4feb870718b1721e5324 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:26:17.535909 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef14b5b-9da8-466a-942e-9ffd2d029f70 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:17.536203 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef14b5b-9da8-466a-942e-9ffd2d029f70 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:17.536490 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef14b5b-9da8-466a-942e-9ffd2d029f70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:17.585841 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-0ef14b5b-9da8-466a-942e-9ffd2d029f70 None None] MFA Rules not processed for user `b04345b7e0dd48d4a93a8be45c187464`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:26:17.588031 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-0ef14b5b-9da8-466a-942e-9ffd2d029f70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:17.664462 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 971/1914] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:26:17 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:26:17.669143 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f105c30b-15a2-49d3-8f57-4a62f1fe212c admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:17.729957 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef14b5b-9da8-466a-942e-9ffd2d029f70 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:17.731339 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-0ef14b5b-9da8-466a-942e-9ffd2d029f70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:17.803280 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f105c30b-15a2-49d3-8f57-4a62f1fe212c admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:17.809238 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9277c3a-6541-435b-9604-cb8260eee86c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:17.809987 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9277c3a-6541-435b-9604-cb8260eee86c admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:17.810359 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9277c3a-6541-435b-9604-cb8260eee86c admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:17.810778 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9277c3a-6541-435b-9604-cb8260eee86c admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:17.811587 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9277c3a-6541-435b-9604-cb8260eee86c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:17.814540 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9277c3a-6541-435b-9604-cb8260eee86c admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:17.843028 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:26:17.855675 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef14b5b-9da8-466a-942e-9ffd2d029f70 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:17.862910 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317c33c5-5aeb-4af0-8975-fb490c44d455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:17.865105 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317c33c5-5aeb-4af0-8975-fb490c44d455 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:17.865696 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317c33c5-5aeb-4af0-8975-fb490c44d455 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:17.866415 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317c33c5-5aeb-4af0-8975-fb490c44d455 admin admin] PATH_INFO: `/v3/projects/71bf4508e4d64f87921b60b2fdced501` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:17.872527 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317c33c5-5aeb-4af0-8975-fb490c44d455 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71bf4508e4d64f87921b60b2fdced501)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:17.874848 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317c33c5-5aeb-4af0-8975-fb490c44d455 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:17.948828 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 944/1915] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:26:17 2022] POST /identity/v3/projects => generated 387 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:26:17.960349 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9277c3a-6541-435b-9604-cb8260eee86c admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:18.007684 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-317c33c5-5aeb-4af0-8975-fb490c44d455 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71bf4508e4d64f87921b60b2fdced501', 'request_id': 'req-317c33c5-5aeb-4af0-8975-fb490c44d455'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:18.015732 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-317c33c5-5aeb-4af0-8975-fb490c44d455 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71bf4508e4d64f87921b60b2fdced501', 'request_id': 'req-317c33c5-5aeb-4af0-8975-fb490c44d455'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:18.038077 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9277c3a-6541-435b-9604-cb8260eee86c admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:18.095861 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34167c1-d881-4cbd-808f-22580b053f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:18.096489 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34167c1-d881-4cbd-808f-22580b053f6c admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:18.096770 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34167c1-d881-4cbd-808f-22580b053f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:18.097039 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34167c1-d881-4cbd-808f-22580b053f6c admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:18.097700 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34167c1-d881-4cbd-808f-22580b053f6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:18.099794 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34167c1-d881-4cbd-808f-22580b053f6c admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:18.101321 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-317c33c5-5aeb-4af0-8975-fb490c44d455 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71bf4508e4d64f87921b60b2fdced501', 'request_id': 'req-317c33c5-5aeb-4af0-8975-fb490c44d455'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:18.116108 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-317c33c5-5aeb-4af0-8975-fb490c44d455 admin admin] The token cache is being invalidate because project 71bf4508e4d64f87921b60b2fdced501 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:26:18.118912 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-317c33c5-5aeb-4af0-8975-fb490c44d455 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:18.118912 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-317c33c5-5aeb-4af0-8975-fb490c44d455 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:18.119349 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 972/1916] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:26:17 2022] DELETE /identity/v3/projects/71bf4508e4d64f87921b60b2fdced501 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:26:18.123699 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f1dd7-d6cb-421b-8f51-2493382c2233 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:18.123982 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f1dd7-d6cb-421b-8f51-2493382c2233 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:18.124257 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f1dd7-d6cb-421b-8f51-2493382c2233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:18.155164 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-ef2f1dd7-d6cb-421b-8f51-2493382c2233 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:26:18.156473 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-ef2f1dd7-d6cb-421b-8f51-2493382c2233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:18.165041 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 973/1917] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:26:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:26:18.181713 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506dc3bf-ce16-4444-8cac-48ec7d3b8d99 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:18.181713 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506dc3bf-ce16-4444-8cac-48ec7d3b8d99 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:18.181713 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506dc3bf-ce16-4444-8cac-48ec7d3b8d99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:18.194730 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-506dc3bf-ce16-4444-8cac-48ec7d3b8d99 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:26:18.195992 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-506dc3bf-ce16-4444-8cac-48ec7d3b8d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:18.205566 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 974/1918] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:26:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:26:18.209056 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 945/1919] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:26:17 2022] POST /identity/v3/users => generated 300 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:26:18.209979 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506dc3bf-ce16-4444-8cac-48ec7d3b8d99 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:18.210132 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34167c1-d881-4cbd-808f-22580b053f6c admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:18.211613 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-b34167c1-d881-4cbd-808f-22580b053f6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:18.220308 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506dc3bf-ce16-4444-8cac-48ec7d3b8d99 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:18.224488 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34167c1-d881-4cbd-808f-22580b053f6c admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:18.227541 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1116276-c517-4b4d-ae5f-836859fe1c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:18.228544 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1116276-c517-4b4d-ae5f-836859fe1c39 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:18.228980 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1116276-c517-4b4d-ae5f-836859fe1c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:18.229309 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9114eb5-e2a0-4060-9080-f1a929b40ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:18.229592 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1116276-c517-4b4d-ae5f-836859fe1c39 admin admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:18.229962 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9114eb5-e2a0-4060-9080-f1a929b40ade admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:18.230288 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9114eb5-e2a0-4060-9080-f1a929b40ade admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:18.230563 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9114eb5-e2a0-4060-9080-f1a929b40ade admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:18.230987 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1116276-c517-4b4d-ae5f-836859fe1c39 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:26:18.231262 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9114eb5-e2a0-4060-9080-f1a929b40ade admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:18.234020 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1116276-c517-4b4d-ae5f-836859fe1c39 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:18.234139 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9114eb5-e2a0-4060-9080-f1a929b40ade admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:18.234239 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1116276-c517-4b4d-ae5f-836859fe1c39 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:18.245533 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 946/1920] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:26:18 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:26:18.249881 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9114eb5-e2a0-4060-9080-f1a929b40ade admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:18.251000 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 975/1921] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Aug 4 21:26:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:26:18.255653 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1116276-c517-4b4d-ae5f-836859fe1c39 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:18.261730 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9114eb5-e2a0-4060-9080-f1a929b40ade admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:18.265914 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1116276-c517-4b4d-ae5f-836859fe1c39 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:18.272547 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea88edf2-1c49-4b1d-9d4c-b94e1216c607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:18.272547 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea88edf2-1c49-4b1d-9d4c-b94e1216c607 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:18.272547 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea88edf2-1c49-4b1d-9d4c-b94e1216c607 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:18.272547 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea88edf2-1c49-4b1d-9d4c-b94e1216c607 admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:18.272898 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d3d274-a0fb-4a31-a9ae-6f7bc5e199a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:18.273035 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea88edf2-1c49-4b1d-9d4c-b94e1216c607 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:18.273556 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d3d274-a0fb-4a31-a9ae-6f7bc5e199a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:18.274304 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d3d274-a0fb-4a31-a9ae-6f7bc5e199a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:18.275004 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea88edf2-1c49-4b1d-9d4c-b94e1216c607 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:18.275165 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d3d274-a0fb-4a31-a9ae-6f7bc5e199a4 admin admin] PATH_INFO: `/v3/projects/75fe1fd5b1ce41fa807401521857ace1/users/c2559e4d6fb3404994809e09ea637293/roles/a432dd4735944ecfab2fa5ce1705d356` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:18.295909 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d3d274-a0fb-4a31-a9ae-6f7bc5e199a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75fe1fd5b1ce41fa807401521857ace1, user_id=c2559e4d6fb3404994809e09ea637293, role_id=a432dd4735944ecfab2fa5ce1705d356)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:18.298927 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d3d274-a0fb-4a31-a9ae-6f7bc5e199a4 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:18.383205 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 976/1922] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:26:18 2022] POST /identity/v3/projects => generated 371 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:26:18.387723 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea88edf2-1c49-4b1d-9d4c-b94e1216c607 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:18.417302 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 947/1923] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:26:18 2022] PUT /identity/v3/projects/75fe1fd5b1ce41fa807401521857ace1/users/c2559e4d6fb3404994809e09ea637293/roles/a432dd4735944ecfab2fa5ce1705d356 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:26:18.422585 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d3d274-a0fb-4a31-a9ae-6f7bc5e199a4 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:18.452560 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea88edf2-1c49-4b1d-9d4c-b94e1216c607 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:18.487452 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d3d274-a0fb-4a31-a9ae-6f7bc5e199a4 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:18.492479 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1baa16-01f7-499b-88a9-2e6515d65a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:18.493111 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1baa16-01f7-499b-88a9-2e6515d65a1e admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:18.493396 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1baa16-01f7-499b-88a9-2e6515d65a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:18.493711 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1baa16-01f7-499b-88a9-2e6515d65a1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:18.494453 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1baa16-01f7-499b-88a9-2e6515d65a1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:18.496131 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1baa16-01f7-499b-88a9-2e6515d65a1e admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:18.508623 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 948/1924] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:26:18 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:26:18.513121 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1baa16-01f7-499b-88a9-2e6515d65a1e admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:18.513721 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bfa385-ecc6-40ef-b3f2-1fa78f51dae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:18.514357 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bfa385-ecc6-40ef-b3f2-1fa78f51dae0 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:18.514641 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bfa385-ecc6-40ef-b3f2-1fa78f51dae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:18.514921 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bfa385-ecc6-40ef-b3f2-1fa78f51dae0 admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:18.515631 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bfa385-ecc6-40ef-b3f2-1fa78f51dae0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:18.518139 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bfa385-ecc6-40ef-b3f2-1fa78f51dae0 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:18.523506 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1baa16-01f7-499b-88a9-2e6515d65a1e admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:18.528480 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e7f6e1-bab9-466c-b141-e325b2fdece4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:18.529221 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e7f6e1-bab9-466c-b141-e325b2fdece4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:18.529452 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e7f6e1-bab9-466c-b141-e325b2fdece4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:18.529725 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e7f6e1-bab9-466c-b141-e325b2fdece4 admin admin] PATH_INFO: `/v3/projects/75fe1fd5b1ce41fa807401521857ace1/users/c2559e4d6fb3404994809e09ea637293/roles/a432dd4735944ecfab2fa5ce1705d356` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:18.531716 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e7f6e1-bab9-466c-b141-e325b2fdece4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75fe1fd5b1ce41fa807401521857ace1, user_id=c2559e4d6fb3404994809e09ea637293, role_id=a432dd4735944ecfab2fa5ce1705d356)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:18.534397 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e7f6e1-bab9-466c-b141-e325b2fdece4 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:18.544754 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 949/1925] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:26:18 2022] PUT /identity/v3/projects/75fe1fd5b1ce41fa807401521857ace1/users/c2559e4d6fb3404994809e09ea637293/roles/a432dd4735944ecfab2fa5ce1705d356 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:26:18.556038 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8136543-37db-4d1f-bde3-ed8a1f243cd0 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:18.556327 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8136543-37db-4d1f-bde3-ed8a1f243cd0 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:18.556607 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8136543-37db-4d1f-bde3-ed8a1f243cd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:18.591898 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-f8136543-37db-4d1f-bde3-ed8a1f243cd0 None None] MFA Rules not processed for user `c2559e4d6fb3404994809e09ea637293`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:26:18.593128 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-f8136543-37db-4d1f-bde3-ed8a1f243cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:18.661582 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 977/1926] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:26:18 2022] POST /identity/v3/users => generated 286 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:26:18.665751 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bfa385-ecc6-40ef-b3f2-1fa78f51dae0 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:18.666191 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 950/1927] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:26:18 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:26:18.677313 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8136543-37db-4d1f-bde3-ed8a1f243cd0 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:18.678768 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-f8136543-37db-4d1f-bde3-ed8a1f243cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:18.680592 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bfa385-ecc6-40ef-b3f2-1fa78f51dae0 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:18.752323 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fd8976-13b1-481e-bea4-e7e048299166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:18.752323 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd8976-13b1-481e-bea4-e7e048299166 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:18.752323 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd8976-13b1-481e-bea4-e7e048299166 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:18.752323 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd8976-13b1-481e-bea4-e7e048299166 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:18.752845 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fd8976-13b1-481e-bea4-e7e048299166 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:18.752845 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fd8976-13b1-481e-bea4-e7e048299166 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:18.756863 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8136543-37db-4d1f-bde3-ed8a1f243cd0 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:18.760763 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ac2400-f3a6-420f-bce0-e06e7c86ed03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e31f2728056b460fa474478eb85ca1d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:18.761071 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 978/1928] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:26:18 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:26:18.762030 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac2400-f3a6-420f-bce0-e06e7c86ed03 service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:18.762167 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac2400-f3a6-420f-bce0-e06e7c86ed03 service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:18.762516 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac2400-f3a6-420f-bce0-e06e7c86ed03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:18.765407 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fd8976-13b1-481e-bea4-e7e048299166 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:18.775979 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ac2400-f3a6-420f-bce0-e06e7c86ed03 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:18.776492 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fd8976-13b1-481e-bea4-e7e048299166 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:18.777774 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ac2400-f3a6-420f-bce0-e06e7c86ed03 service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:18.783449 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26127bc9-b0df-46af-a46e-ef8325de4741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:18.783449 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26127bc9-b0df-46af-a46e-ef8325de4741 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:18.783449 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26127bc9-b0df-46af-a46e-ef8325de4741 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:18.783449 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26127bc9-b0df-46af-a46e-ef8325de4741 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:18.784759 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26127bc9-b0df-46af-a46e-ef8325de4741 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:18.785817 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26127bc9-b0df-46af-a46e-ef8325de4741 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:18.792066 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 951/1929] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:26:18 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:26:18.804115 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 979/1930] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:26:18 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:26:18.810098 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ac2400-f3a6-420f-bce0-e06e7c86ed03 service cinder] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:18.814670 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:26:18.824464 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ac2400-f3a6-420f-bce0-e06e7c86ed03 service cinder] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:18.829246 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc4584d-3b2f-4700-975f-8fa738bdef1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:18.829911 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc4584d-3b2f-4700-975f-8fa738bdef1b admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:18.830211 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc4584d-3b2f-4700-975f-8fa738bdef1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:18.830482 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc4584d-3b2f-4700-975f-8fa738bdef1b admin admin] PATH_INFO: `/v3/projects/46d62342f37d4d5aa77d4a42fb3cda65/users/5296a0af07f14a7e903f2fba21d32276/roles/8feebc60363c4feb870718b1721e5324` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:18.845684 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc4584d-3b2f-4700-975f-8fa738bdef1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=46d62342f37d4d5aa77d4a42fb3cda65, user_id=5296a0af07f14a7e903f2fba21d32276, role_id=8feebc60363c4feb870718b1721e5324)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:18.848640 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc4584d-3b2f-4700-975f-8fa738bdef1b admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:18.951934 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:DELETE=1,keystone:SELECT=95,keystone:INSERT=4 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:26:18.952157 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 952/1931] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:26:18 2022] PUT /identity/v3/projects/46d62342f37d4d5aa77d4a42fb3cda65/users/5296a0af07f14a7e903f2fba21d32276/roles/8feebc60363c4feb870718b1721e5324 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:26:18.955526 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b92db8-b3d3-4b34-83fa-0c8d2a693211 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:18.958723 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b92db8-b3d3-4b34-83fa-0c8d2a693211 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:18.958723 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b92db8-b3d3-4b34-83fa-0c8d2a693211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:18.981429 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-72b92db8-b3d3-4b34-83fa-0c8d2a693211 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:26:18.986854 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-72b92db8-b3d3-4b34-83fa-0c8d2a693211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:19.088204 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 980/1932] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:26:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:26:19.099341 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc4584d-3b2f-4700-975f-8fa738bdef1b admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:19.101157 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-9cc4584d-3b2f-4700-975f-8fa738bdef1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:19.161158 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc4584d-3b2f-4700-975f-8fa738bdef1b admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:19.164875 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a2ceb2-f7b4-4dc9-980f-6c6b422dfe21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:19.165575 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a2ceb2-f7b4-4dc9-980f-6c6b422dfe21 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:19.165893 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a2ceb2-f7b4-4dc9-980f-6c6b422dfe21 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:19.166147 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a2ceb2-f7b4-4dc9-980f-6c6b422dfe21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:19.176700 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a2ceb2-f7b4-4dc9-980f-6c6b422dfe21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:19.178251 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a2ceb2-f7b4-4dc9-980f-6c6b422dfe21 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:19.188137 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 953/1933] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:26:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:26:20.130442 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b92db8-b3d3-4b34-83fa-0c8d2a693211 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:20.140631 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b92db8-b3d3-4b34-83fa-0c8d2a693211 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:20.146802 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468b7f0a-4c77-4758-8edc-3365f1b41e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:20.147532 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468b7f0a-4c77-4758-8edc-3365f1b41e02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:20.147823 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468b7f0a-4c77-4758-8edc-3365f1b41e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:20.148107 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468b7f0a-4c77-4758-8edc-3365f1b41e02 admin admin] PATH_INFO: `/v3/users/b04345b7e0dd48d4a93a8be45c187464` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:20.149446 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468b7f0a-4c77-4758-8edc-3365f1b41e02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b04345b7e0dd48d4a93a8be45c187464)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:20.151593 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468b7f0a-4c77-4758-8edc-3365f1b41e02 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:20.187304 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:26:20.420755 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-468b7f0a-4c77-4758-8edc-3365f1b41e02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b04345b7e0dd48d4a93a8be45c187464'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:20.434055 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-468b7f0a-4c77-4758-8edc-3365f1b41e02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b04345b7e0dd48d4a93a8be45c187464'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:20.434791 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-468b7f0a-4c77-4758-8edc-3365f1b41e02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b04345b7e0dd48d4a93a8be45c187464'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:20.435707 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-468b7f0a-4c77-4758-8edc-3365f1b41e02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b04345b7e0dd48d4a93a8be45c187464'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:20.546897 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-468b7f0a-4c77-4758-8edc-3365f1b41e02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b04345b7e0dd48d4a93a8be45c187464'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:20.561044 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 981/1934] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:26:20 2022] DELETE /identity/v3/users/b04345b7e0dd48d4a93a8be45c187464 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:26:20.566702 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa257e5-4597-4999-929d-9dc0f6d16388 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:20.566924 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa257e5-4597-4999-929d-9dc0f6d16388 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:20.567184 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa257e5-4597-4999-929d-9dc0f6d16388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:20.586366 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-3aa257e5-4597-4999-929d-9dc0f6d16388 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:26:20.587919 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-3aa257e5-4597-4999-929d-9dc0f6d16388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:20.653202 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 954/1935] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:26:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:26:20.670016 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468b7f0a-4c77-4758-8edc-3365f1b41e02 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:20.679573 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-468b7f0a-4c77-4758-8edc-3365f1b41e02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:20.733808 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468b7f0a-4c77-4758-8edc-3365f1b41e02 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:20.738492 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a1e28b-e9b7-4256-bc2d-61c1db95dfc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:20.739224 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a1e28b-e9b7-4256-bc2d-61c1db95dfc6 service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:20.739536 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a1e28b-e9b7-4256-bc2d-61c1db95dfc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:20.739817 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a1e28b-e9b7-4256-bc2d-61c1db95dfc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:20.745362 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36d888d-ccbb-4d72-b193-f36a71204c47 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:20.745759 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36d888d-ccbb-4d72-b193-f36a71204c47 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:20.746140 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36d888d-ccbb-4d72-b193-f36a71204c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:20.752077 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a1e28b-e9b7-4256-bc2d-61c1db95dfc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:20.754184 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a1e28b-e9b7-4256-bc2d-61c1db95dfc6 service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:20.766311 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 982/1936] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:26:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:26:20.785046 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-f36d888d-ccbb-4d72-b193-f36a71204c47 None None] MFA Rules not processed for user `5296a0af07f14a7e903f2fba21d32276`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:26:20.786214 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-f36d888d-ccbb-4d72-b193-f36a71204c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:20.879227 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 955/1937] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:26:20 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:26:20.884241 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a1e28b-e9b7-4256-bc2d-61c1db95dfc6 service neutron] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:20.885935 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-06a1e28b-e9b7-4256-bc2d-61c1db95dfc6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:20.898914 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a1e28b-e9b7-4256-bc2d-61c1db95dfc6 service neutron] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:20.904481 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5510b3db-705b-4139-a3e4-e8eb52e7c1e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:20.905109 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5510b3db-705b-4139-a3e4-e8eb52e7c1e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:20.905414 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5510b3db-705b-4139-a3e4-e8eb52e7c1e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:20.905696 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5510b3db-705b-4139-a3e4-e8eb52e7c1e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:20.906398 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5510b3db-705b-4139-a3e4-e8eb52e7c1e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:20.908867 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5510b3db-705b-4139-a3e4-e8eb52e7c1e1 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:21.100433 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 983/1938] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:26:20 2022] POST /identity/v3/projects => generated 371 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:26:21.101829 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36d888d-ccbb-4d72-b193-f36a71204c47 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:21.114371 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36d888d-ccbb-4d72-b193-f36a71204c47 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:21.181920 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21570714-03af-48a4-918f-bb1221b29211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:21.182555 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21570714-03af-48a4-918f-bb1221b29211 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:21.182840 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21570714-03af-48a4-918f-bb1221b29211 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:21.183121 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21570714-03af-48a4-918f-bb1221b29211 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:21.183831 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21570714-03af-48a4-918f-bb1221b29211 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:21.185800 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21570714-03af-48a4-918f-bb1221b29211 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:21.230927 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:26:21.364919 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 956/1939] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:26:21 2022] POST /identity/v3/users => generated 292 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:26:21.370086 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5510b3db-705b-4139-a3e4-e8eb52e7c1e1 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:21.381816 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5510b3db-705b-4139-a3e4-e8eb52e7c1e1 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:21.386888 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eb0c0a-6c95-4991-bcc6-106d5b9ca027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:21.387568 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eb0c0a-6c95-4991-bcc6-106d5b9ca027 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:21.387851 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eb0c0a-6c95-4991-bcc6-106d5b9ca027 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:21.388126 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eb0c0a-6c95-4991-bcc6-106d5b9ca027 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:21.388839 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6eb0c0a-6c95-4991-bcc6-106d5b9ca027 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:21.390494 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6eb0c0a-6c95-4991-bcc6-106d5b9ca027 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:21.408297 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 984/1940] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:26:21 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:26:21.409701 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21570714-03af-48a4-918f-bb1221b29211 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:21.421826 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21570714-03af-48a4-918f-bb1221b29211 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:21.427921 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260f955a-2946-4277-9b2d-49b366008ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:21.428623 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260f955a-2946-4277-9b2d-49b366008ad2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:21.428934 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260f955a-2946-4277-9b2d-49b366008ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:21.429366 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260f955a-2946-4277-9b2d-49b366008ad2 admin admin] PATH_INFO: `/v3/projects/9b566d7b2e6b49178a9afd6a9fc0e86a/users/b522296f29544acf9377a7b52314a6b8/roles/a432dd4735944ecfab2fa5ce1705d356` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:21.450169 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260f955a-2946-4277-9b2d-49b366008ad2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b566d7b2e6b49178a9afd6a9fc0e86a, user_id=b522296f29544acf9377a7b52314a6b8, role_id=a432dd4735944ecfab2fa5ce1705d356)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:21.453109 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260f955a-2946-4277-9b2d-49b366008ad2 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:21.590943 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eb0c0a-6c95-4991-bcc6-106d5b9ca027 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:21.594780 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-a6eb0c0a-6c95-4991-bcc6-106d5b9ca027 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:21.616272 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eb0c0a-6c95-4991-bcc6-106d5b9ca027 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:21.631889 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 957/1941] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:26:21 2022] PUT /identity/v3/projects/9b566d7b2e6b49178a9afd6a9fc0e86a/users/b522296f29544acf9377a7b52314a6b8/roles/a432dd4735944ecfab2fa5ce1705d356 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:26:21.634486 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260f955a-2946-4277-9b2d-49b366008ad2 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:21.706546 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749f397d-a95e-4a6a-aa69-89cf097b95a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:21.707204 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749f397d-a95e-4a6a-aa69-89cf097b95a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:21.707510 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749f397d-a95e-4a6a-aa69-89cf097b95a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:21.707783 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749f397d-a95e-4a6a-aa69-89cf097b95a2 admin admin] PATH_INFO: `/v3/projects/c4113b1df3374dca9515c8ec986688b1` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:21.708941 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749f397d-a95e-4a6a-aa69-89cf097b95a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4113b1df3374dca9515c8ec986688b1)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:21.710898 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749f397d-a95e-4a6a-aa69-89cf097b95a2 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:21.725429 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260f955a-2946-4277-9b2d-49b366008ad2 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:21.730594 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f14f70-d1c1-42e8-b305-b4983c3e7c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:21.731260 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f14f70-d1c1-42e8-b305-b4983c3e7c3d admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:21.731568 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f14f70-d1c1-42e8-b305-b4983c3e7c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:21.731836 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f14f70-d1c1-42e8-b305-b4983c3e7c3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:21.732561 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f14f70-d1c1-42e8-b305-b4983c3e7c3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:21.734183 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f14f70-d1c1-42e8-b305-b4983c3e7c3d admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:21.744971 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 958/1942] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:26:21 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:26:21.749208 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f14f70-d1c1-42e8-b305-b4983c3e7c3d admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:21.761089 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f14f70-d1c1-42e8-b305-b4983c3e7c3d admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:21.765271 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492a93cd-6f1a-47fd-baf8-b77545a33a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:21.766581 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492a93cd-6f1a-47fd-baf8-b77545a33a1b admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:21.766865 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492a93cd-6f1a-47fd-baf8-b77545a33a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:21.767186 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492a93cd-6f1a-47fd-baf8-b77545a33a1b admin admin] PATH_INFO: `/v3/projects/9b566d7b2e6b49178a9afd6a9fc0e86a/users/b522296f29544acf9377a7b52314a6b8/roles/a432dd4735944ecfab2fa5ce1705d356` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:21.769469 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492a93cd-6f1a-47fd-baf8-b77545a33a1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b566d7b2e6b49178a9afd6a9fc0e86a, user_id=b522296f29544acf9377a7b52314a6b8, role_id=a432dd4735944ecfab2fa5ce1705d356)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:21.772184 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492a93cd-6f1a-47fd-baf8-b77545a33a1b admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:21.790432 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 959/1943] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:26:21 2022] PUT /identity/v3/projects/9b566d7b2e6b49178a9afd6a9fc0e86a/users/b522296f29544acf9377a7b52314a6b8/roles/a432dd4735944ecfab2fa5ce1705d356 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:26:21.856090 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-749f397d-a95e-4a6a-aa69-89cf097b95a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4113b1df3374dca9515c8ec986688b1', 'request_id': 'req-749f397d-a95e-4a6a-aa69-89cf097b95a2'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:21.862585 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-749f397d-a95e-4a6a-aa69-89cf097b95a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4113b1df3374dca9515c8ec986688b1', 'request_id': 'req-749f397d-a95e-4a6a-aa69-89cf097b95a2'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:21.992120 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-749f397d-a95e-4a6a-aa69-89cf097b95a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4113b1df3374dca9515c8ec986688b1', 'request_id': 'req-749f397d-a95e-4a6a-aa69-89cf097b95a2'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:22.005122 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-749f397d-a95e-4a6a-aa69-89cf097b95a2 admin admin] The token cache is being invalidate because project c4113b1df3374dca9515c8ec986688b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:26:22.005421 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-749f397d-a95e-4a6a-aa69-89cf097b95a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:22.006079 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-749f397d-a95e-4a6a-aa69-89cf097b95a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:22.007753 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 985/1944] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:26:21 2022] DELETE /identity/v3/projects/c4113b1df3374dca9515c8ec986688b1 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:26:22.011925 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492a93cd-6f1a-47fd-baf8-b77545a33a1b admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:22.014138 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-492a93cd-6f1a-47fd-baf8-b77545a33a1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:22.082975 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492a93cd-6f1a-47fd-baf8-b77545a33a1b admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:22.140752 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b1ff7f-af0f-432d-a82b-be9f2f085213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:22.141450 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b1ff7f-af0f-432d-a82b-be9f2f085213 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:22.141718 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b1ff7f-af0f-432d-a82b-be9f2f085213 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:22.141992 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b1ff7f-af0f-432d-a82b-be9f2f085213 admin admin] PATH_INFO: `/v3/users/c2559e4d6fb3404994809e09ea637293` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:22.143146 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b1ff7f-af0f-432d-a82b-be9f2f085213 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2559e4d6fb3404994809e09ea637293)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:22.145133 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b1ff7f-af0f-432d-a82b-be9f2f085213 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:22.435258 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-d9b1ff7f-af0f-432d-a82b-be9f2f085213 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2559e4d6fb3404994809e09ea637293'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:22.448645 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-d9b1ff7f-af0f-432d-a82b-be9f2f085213 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2559e4d6fb3404994809e09ea637293'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:22.452717 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-d9b1ff7f-af0f-432d-a82b-be9f2f085213 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2559e4d6fb3404994809e09ea637293'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:22.456773 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-d9b1ff7f-af0f-432d-a82b-be9f2f085213 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2559e4d6fb3404994809e09ea637293'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:22.457460 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-d9b1ff7f-af0f-432d-a82b-be9f2f085213 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2559e4d6fb3404994809e09ea637293'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:22.550614 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:26:22.617091 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 960/1945] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:26:22 2022] DELETE /identity/v3/users/c2559e4d6fb3404994809e09ea637293 => generated 0 bytes in 606 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:26:22.730576 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:26:23.132985 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749f397d-a95e-4a6a-aa69-89cf097b95a2 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:23.182619 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749f397d-a95e-4a6a-aa69-89cf097b95a2 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:23.279440 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c813a685-6d46-4aec-b38a-3c11e4883eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:23.280210 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c813a685-6d46-4aec-b38a-3c11e4883eae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:23.280446 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c813a685-6d46-4aec-b38a-3c11e4883eae admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:23.280725 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c813a685-6d46-4aec-b38a-3c11e4883eae admin admin] PATH_INFO: `/v3/projects/75fe1fd5b1ce41fa807401521857ace1` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:23.282349 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c813a685-6d46-4aec-b38a-3c11e4883eae admin admin] RBAC: Authorizing `identity:delete_project(project_id=75fe1fd5b1ce41fa807401521857ace1)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:23.285398 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c813a685-6d46-4aec-b38a-3c11e4883eae admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:23.397789 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-c813a685-6d46-4aec-b38a-3c11e4883eae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75fe1fd5b1ce41fa807401521857ace1', 'request_id': 'req-c813a685-6d46-4aec-b38a-3c11e4883eae'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:23.403347 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-c813a685-6d46-4aec-b38a-3c11e4883eae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75fe1fd5b1ce41fa807401521857ace1', 'request_id': 'req-c813a685-6d46-4aec-b38a-3c11e4883eae'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:23.478976 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-c813a685-6d46-4aec-b38a-3c11e4883eae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75fe1fd5b1ce41fa807401521857ace1', 'request_id': 'req-c813a685-6d46-4aec-b38a-3c11e4883eae'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:23.491530 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-c813a685-6d46-4aec-b38a-3c11e4883eae admin admin] The token cache is being invalidate because project 75fe1fd5b1ce41fa807401521857ace1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:26:23.491965 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-c813a685-6d46-4aec-b38a-3c11e4883eae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:23.492655 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-c813a685-6d46-4aec-b38a-3c11e4883eae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:23.494522 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 986/1946] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:26:23 2022] DELETE /identity/v3/projects/75fe1fd5b1ce41fa807401521857ace1 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:26:23.499704 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dbbe11-5950-4d11-9db7-d3d398cd419e None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:23.501218 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dbbe11-5950-4d11-9db7-d3d398cd419e None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:23.501541 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dbbe11-5950-4d11-9db7-d3d398cd419e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:23.527764 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-c2dbbe11-5950-4d11-9db7-d3d398cd419e None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:26:23.534778 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-c2dbbe11-5950-4d11-9db7-d3d398cd419e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:23.534947 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17702379-04d0-4437-b13d-fd040cc797cf None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:23.534947 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17702379-04d0-4437-b13d-fd040cc797cf None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:23.534947 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17702379-04d0-4437-b13d-fd040cc797cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:23.566307 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-17702379-04d0-4437-b13d-fd040cc797cf None None] MFA Rules not processed for user `b522296f29544acf9377a7b52314a6b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:26:23.569047 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-17702379-04d0-4437-b13d-fd040cc797cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:23.608305 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 961/1947] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:26:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:26:23.616474 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd51221-2760-4238-9eff-80864213a588 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:23.616766 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd51221-2760-4238-9eff-80864213a588 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:23.617044 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd51221-2760-4238-9eff-80864213a588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:23.641823 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-1bd51221-2760-4238-9eff-80864213a588 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:26:23.648139 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-1bd51221-2760-4238-9eff-80864213a588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:23.652158 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 962/1948] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:26:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:26:23.656382 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 987/1949] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:26:23 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:26:23.659506 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd51221-2760-4238-9eff-80864213a588 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:23.668650 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17702379-04d0-4437-b13d-fd040cc797cf None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:23.670240 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd51221-2760-4238-9eff-80864213a588 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:23.670786 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-17702379-04d0-4437-b13d-fd040cc797cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:23.676207 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e486bed5-8f77-4273-bd76-850c1051e345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:23.676844 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e486bed5-8f77-4273-bd76-850c1051e345 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:23.677131 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e486bed5-8f77-4273-bd76-850c1051e345 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:23.695298 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e486bed5-8f77-4273-bd76-850c1051e345 admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:23.695298 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e486bed5-8f77-4273-bd76-850c1051e345 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:26:23.695298 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e486bed5-8f77-4273-bd76-850c1051e345 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:23.695298 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e486bed5-8f77-4273-bd76-850c1051e345 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:23.695298 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 963/1950] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Aug 4 21:26:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:26:23.698669 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e486bed5-8f77-4273-bd76-850c1051e345 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:23.709138 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e486bed5-8f77-4273-bd76-850c1051e345 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:23.715177 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81291e93-a80c-42df-ba17-4cafd58570a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:23.715858 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81291e93-a80c-42df-ba17-4cafd58570a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:23.724172 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81291e93-a80c-42df-ba17-4cafd58570a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:23.724172 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81291e93-a80c-42df-ba17-4cafd58570a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:23.724172 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81291e93-a80c-42df-ba17-4cafd58570a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:23.724172 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81291e93-a80c-42df-ba17-4cafd58570a5 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:23.740125 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17702379-04d0-4437-b13d-fd040cc797cf None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:23.743904 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3416e4f1-1c2f-46e3-a986-14fb67aa7173 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e31f2728056b460fa474478eb85ca1d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:23.745307 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3416e4f1-1c2f-46e3-a986-14fb67aa7173 service cinder] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:23.745598 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3416e4f1-1c2f-46e3-a986-14fb67aa7173 service cinder] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:23.745879 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3416e4f1-1c2f-46e3-a986-14fb67aa7173 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:23.747797 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-3416e4f1-1c2f-46e3-a986-14fb67aa7173 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:23.805817 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 964/1951] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:26:23 2022] POST /identity/v3/projects => generated 383 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:26:23.810054 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81291e93-a80c-42df-ba17-4cafd58570a5 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:23.812236 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3416e4f1-1c2f-46e3-a986-14fb67aa7173 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:23.814690 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3416e4f1-1c2f-46e3-a986-14fb67aa7173 service cinder] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:23.825271 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 988/1952] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:26:23 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:26:23.872885 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:26:23.894457 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81291e93-a80c-42df-ba17-4cafd58570a5 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:23.899634 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9284fd-1e69-4500-826b-2faa2f83bf25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:23.900524 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9284fd-1e69-4500-826b-2faa2f83bf25 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:23.901224 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9284fd-1e69-4500-826b-2faa2f83bf25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:23.901750 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9284fd-1e69-4500-826b-2faa2f83bf25 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:23.902588 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9284fd-1e69-4500-826b-2faa2f83bf25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:23.904773 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9284fd-1e69-4500-826b-2faa2f83bf25 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:24.045103 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 965/1953] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:26:23 2022] POST /identity/v3/users => generated 292 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:26:24.050237 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3416e4f1-1c2f-46e3-a986-14fb67aa7173 service cinder] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:24.061324 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3416e4f1-1c2f-46e3-a986-14fb67aa7173 service cinder] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:24.070196 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1242ddf9-f21e-4723-82ab-face8bf31518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:24.071504 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1242ddf9-f21e-4723-82ab-face8bf31518 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:24.071504 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1242ddf9-f21e-4723-82ab-face8bf31518 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:24.071504 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1242ddf9-f21e-4723-82ab-face8bf31518 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:24.072174 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1242ddf9-f21e-4723-82ab-face8bf31518 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:24.073811 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1242ddf9-f21e-4723-82ab-face8bf31518 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:24.087918 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 989/1954] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:26:24 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:26:24.089605 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9284fd-1e69-4500-826b-2faa2f83bf25 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:24.100794 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9284fd-1e69-4500-826b-2faa2f83bf25 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:24.106609 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4c1ee3-a286-454f-9caa-782a6ab53515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:24.107300 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4c1ee3-a286-454f-9caa-782a6ab53515 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:24.107647 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4c1ee3-a286-454f-9caa-782a6ab53515 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:24.107927 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4c1ee3-a286-454f-9caa-782a6ab53515 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:24.109272 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4c1ee3-a286-454f-9caa-782a6ab53515 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:24.111468 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4c1ee3-a286-454f-9caa-782a6ab53515 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:24.122763 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 966/1955] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:26:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:26:24.136744 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1242ddf9-f21e-4723-82ab-face8bf31518 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:24.147238 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1242ddf9-f21e-4723-82ab-face8bf31518 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:24.152466 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:26:24.152980 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8c46ed-3fd3-470e-a385-e97c75b55beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:24.154132 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8c46ed-3fd3-470e-a385-e97c75b55beb admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:24.154599 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8c46ed-3fd3-470e-a385-e97c75b55beb admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:24.155059 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8c46ed-3fd3-470e-a385-e97c75b55beb admin admin] PATH_INFO: `/v3/projects/99addd712333446abbe3593b715d1e4a/users/c486f446b0324eae9a47b9249b45d4a4/roles/8feebc60363c4feb870718b1721e5324` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:24.177100 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8c46ed-3fd3-470e-a385-e97c75b55beb admin admin] RBAC: Authorizing `identity:create_grant(project_id=99addd712333446abbe3593b715d1e4a, user_id=c486f446b0324eae9a47b9249b45d4a4, role_id=8feebc60363c4feb870718b1721e5324)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:24.179887 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8c46ed-3fd3-470e-a385-e97c75b55beb admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:24.278603 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 990/1956] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:26:24 2022] PUT /identity/v3/projects/99addd712333446abbe3593b715d1e4a/users/c486f446b0324eae9a47b9249b45d4a4/roles/8feebc60363c4feb870718b1721e5324 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:26:24.290094 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8934d092-ee02-4d7e-9020-bbbc49b38e4e None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:24.290380 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8934d092-ee02-4d7e-9020-bbbc49b38e4e None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:24.290659 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8934d092-ee02-4d7e-9020-bbbc49b38e4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:24.308003 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:26:24.325576 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-8934d092-ee02-4d7e-9020-bbbc49b38e4e None None] MFA Rules not processed for user `c486f446b0324eae9a47b9249b45d4a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:26:24.327204 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-8934d092-ee02-4d7e-9020-bbbc49b38e4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:24.392535 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 967/1957] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:26:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:26:24.397059 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8c46ed-3fd3-470e-a385-e97c75b55beb admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:24.464171 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8c46ed-3fd3-470e-a385-e97c75b55beb admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:24.470153 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbd0309-27e1-45e3-9023-a6b9eeee4b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:24.470793 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbd0309-27e1-45e3-9023-a6b9eeee4b6e admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:24.471079 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbd0309-27e1-45e3-9023-a6b9eeee4b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:24.471380 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbd0309-27e1-45e3-9023-a6b9eeee4b6e admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:24.472238 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbd0309-27e1-45e3-9023-a6b9eeee4b6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:24.476902 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbd0309-27e1-45e3-9023-a6b9eeee4b6e admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:24.600433 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 991/1958] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:26:24 2022] POST /identity/v3/projects => generated 383 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:26:24.604870 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8934d092-ee02-4d7e-9020-bbbc49b38e4e None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:24.617983 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8934d092-ee02-4d7e-9020-bbbc49b38e4e None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:24.682464 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7558fb4c-bb85-411d-9af9-17349c28a26a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:24.689747 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7558fb4c-bb85-411d-9af9-17349c28a26a admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:24.689747 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7558fb4c-bb85-411d-9af9-17349c28a26a admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:24.689747 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7558fb4c-bb85-411d-9af9-17349c28a26a admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:24.689747 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7558fb4c-bb85-411d-9af9-17349c28a26a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:24.689747 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7558fb4c-bb85-411d-9af9-17349c28a26a admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:24.820470 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 968/1959] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:26:24 2022] POST /identity/v3/users => generated 298 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:26:24.824699 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbd0309-27e1-45e3-9023-a6b9eeee4b6e admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:24.836216 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbd0309-27e1-45e3-9023-a6b9eeee4b6e admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:24.841041 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4a96ef-514f-4d8d-902e-2852d93f613e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:24.841680 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4a96ef-514f-4d8d-902e-2852d93f613e admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:24.841979 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4a96ef-514f-4d8d-902e-2852d93f613e admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:24.842253 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4a96ef-514f-4d8d-902e-2852d93f613e admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:24.842951 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4a96ef-514f-4d8d-902e-2852d93f613e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:24.845190 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4a96ef-514f-4d8d-902e-2852d93f613e admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:24.856201 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 992/1960] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:26:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:26:24.864816 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7558fb4c-bb85-411d-9af9-17349c28a26a admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:24.875262 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7558fb4c-bb85-411d-9af9-17349c28a26a admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:24.880127 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cf333c-84fd-4d0a-9edb-366f533d81d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:24.880804 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cf333c-84fd-4d0a-9edb-366f533d81d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:24.881146 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cf333c-84fd-4d0a-9edb-366f533d81d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:24.881438 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cf333c-84fd-4d0a-9edb-366f533d81d3 admin admin] PATH_INFO: `/v3/projects/7d40d66b6f5d4c64bee2257b7fd35b84/users/b6959c4393be4ad1ae22133707a704a6/roles/a432dd4735944ecfab2fa5ce1705d356` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:24.898407 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cf333c-84fd-4d0a-9edb-366f533d81d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d40d66b6f5d4c64bee2257b7fd35b84, user_id=b6959c4393be4ad1ae22133707a704a6, role_id=a432dd4735944ecfab2fa5ce1705d356)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:24.902286 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cf333c-84fd-4d0a-9edb-366f533d81d3 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:24.993897 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 969/1961] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:26:24 2022] PUT /identity/v3/projects/7d40d66b6f5d4c64bee2257b7fd35b84/users/b6959c4393be4ad1ae22133707a704a6/roles/a432dd4735944ecfab2fa5ce1705d356 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:26:24.997874 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4a96ef-514f-4d8d-902e-2852d93f613e admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:25.008588 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4a96ef-514f-4d8d-902e-2852d93f613e admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:25.074365 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ce7298-5186-4d35-8124-d7c029dbef2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:25.074991 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ce7298-5186-4d35-8124-d7c029dbef2f admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:25.075274 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ce7298-5186-4d35-8124-d7c029dbef2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:25.075617 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ce7298-5186-4d35-8124-d7c029dbef2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:25.076311 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ce7298-5186-4d35-8124-d7c029dbef2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:25.077979 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ce7298-5186-4d35-8124-d7c029dbef2f admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:25.093613 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 993/1962] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:26:24 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:26:25.099919 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cf333c-84fd-4d0a-9edb-366f533d81d3 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:25.116053 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cf333c-84fd-4d0a-9edb-366f533d81d3 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:25.121736 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c63931-c198-4585-8aae-b0a95253ee02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:25.123783 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c63931-c198-4585-8aae-b0a95253ee02 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:25.123783 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c63931-c198-4585-8aae-b0a95253ee02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:25.123783 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c63931-c198-4585-8aae-b0a95253ee02 admin admin] PATH_INFO: `/v3/projects/7d40d66b6f5d4c64bee2257b7fd35b84/users/b6959c4393be4ad1ae22133707a704a6/roles/a432dd4735944ecfab2fa5ce1705d356` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:25.125138 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c63931-c198-4585-8aae-b0a95253ee02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d40d66b6f5d4c64bee2257b7fd35b84, user_id=b6959c4393be4ad1ae22133707a704a6, role_id=a432dd4735944ecfab2fa5ce1705d356)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:25.127877 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c63931-c198-4585-8aae-b0a95253ee02 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:25.139806 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 970/1963] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:26:25 2022] PUT /identity/v3/projects/7d40d66b6f5d4c64bee2257b7fd35b84/users/b6959c4393be4ad1ae22133707a704a6/roles/a432dd4735944ecfab2fa5ce1705d356 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:26:25.152767 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be73d10-dfcd-4835-b39e-aa733851af7d None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:25.153072 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be73d10-dfcd-4835-b39e-aa733851af7d None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:25.153409 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be73d10-dfcd-4835-b39e-aa733851af7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:25.189436 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-4be73d10-dfcd-4835-b39e-aa733851af7d None None] MFA Rules not processed for user `b6959c4393be4ad1ae22133707a704a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:26:25.190819 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-4be73d10-dfcd-4835-b39e-aa733851af7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:25.273656 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 994/1964] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:26:25 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:26:25.286633 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c63931-c198-4585-8aae-b0a95253ee02 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:25.288576 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-b3c63931-c198-4585-8aae-b0a95253ee02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:25.348402 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:26:25.360078 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c63931-c198-4585-8aae-b0a95253ee02 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:25.363789 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828db730-f8e4-4bed-a31b-0856b49d2101 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e31f2728056b460fa474478eb85ca1d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:25.364470 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828db730-f8e4-4bed-a31b-0856b49d2101 service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:25.364688 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828db730-f8e4-4bed-a31b-0856b49d2101 service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:25.364961 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828db730-f8e4-4bed-a31b-0856b49d2101 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:25.379669 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828db730-f8e4-4bed-a31b-0856b49d2101 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:25.381419 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828db730-f8e4-4bed-a31b-0856b49d2101 service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:25.403776 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 971/1965] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:26:25 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:26:25.545388 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be73d10-dfcd-4835-b39e-aa733851af7d None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:25.555572 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be73d10-dfcd-4835-b39e-aa733851af7d None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:25.559416 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45640230-d2cb-4935-b862-39fde2c95670 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e31f2728056b460fa474478eb85ca1d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:25.560049 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45640230-d2cb-4935-b862-39fde2c95670 service cinder] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:25.560334 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45640230-d2cb-4935-b862-39fde2c95670 service cinder] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:25.560611 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45640230-d2cb-4935-b862-39fde2c95670 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:25.628565 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45640230-d2cb-4935-b862-39fde2c95670 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:25.630140 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45640230-d2cb-4935-b862-39fde2c95670 service cinder] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:25.639927 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 995/1966] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:26:25 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:26:25.868336 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:26:26.216976 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:26:26.242834 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828db730-f8e4-4bed-a31b-0856b49d2101 service cinder] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:26.310012 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828db730-f8e4-4bed-a31b-0856b49d2101 service cinder] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:26.316047 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ba35c4-e0c6-4ec3-94fd-eb78737705da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:26.317052 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ba35c4-e0c6-4ec3-94fd-eb78737705da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:26.317450 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ba35c4-e0c6-4ec3-94fd-eb78737705da admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:26.317868 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ba35c4-e0c6-4ec3-94fd-eb78737705da admin admin] PATH_INFO: `/v3/users/c486f446b0324eae9a47b9249b45d4a4` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:26.319075 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ba35c4-e0c6-4ec3-94fd-eb78737705da admin admin] RBAC: Authorizing `identity:delete_user(user_id=c486f446b0324eae9a47b9249b45d4a4)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:26.321104 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ba35c4-e0c6-4ec3-94fd-eb78737705da admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:26.518024 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-40ba35c4-e0c6-4ec3-94fd-eb78737705da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c486f446b0324eae9a47b9249b45d4a4'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:26.539170 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-40ba35c4-e0c6-4ec3-94fd-eb78737705da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c486f446b0324eae9a47b9249b45d4a4'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:26.540085 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-40ba35c4-e0c6-4ec3-94fd-eb78737705da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c486f446b0324eae9a47b9249b45d4a4'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:26.548660 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-40ba35c4-e0c6-4ec3-94fd-eb78737705da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c486f446b0324eae9a47b9249b45d4a4'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:26.549486 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-40ba35c4-e0c6-4ec3-94fd-eb78737705da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c486f446b0324eae9a47b9249b45d4a4'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:26.691185 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 972/1967] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:26:26 2022] DELETE /identity/v3/users/c486f446b0324eae9a47b9249b45d4a4 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:26:26.695824 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3215ba-364c-4e1f-aa64-1fc4162a182b None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:26.696140 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3215ba-364c-4e1f-aa64-1fc4162a182b None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:26.696417 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3215ba-364c-4e1f-aa64-1fc4162a182b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:26.715888 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-1a3215ba-364c-4e1f-aa64-1fc4162a182b None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:26:26.717105 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-1a3215ba-364c-4e1f-aa64-1fc4162a182b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:26.781233 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ba35c4-e0c6-4ec3-94fd-eb78737705da admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:26.783130 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-40ba35c4-e0c6-4ec3-94fd-eb78737705da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:26.787181 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 996/1968] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:26:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:26:26.801790 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3215ba-364c-4e1f-aa64-1fc4162a182b None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:26.803285 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-1a3215ba-364c-4e1f-aa64-1fc4162a182b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:26.858231 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ba35c4-e0c6-4ec3-94fd-eb78737705da admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:26.864265 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f32ed6-3e71-4f58-97ae-460738ad528d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e31f2728056b460fa474478eb85ca1d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:26.865217 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f32ed6-3e71-4f58-97ae-460738ad528d service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:26.865563 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f32ed6-3e71-4f58-97ae-460738ad528d service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:26.865953 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f32ed6-3e71-4f58-97ae-460738ad528d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:26.868520 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-25f32ed6-3e71-4f58-97ae-460738ad528d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:26.873090 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3215ba-364c-4e1f-aa64-1fc4162a182b None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:26.878712 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e302d5c-94d0-43d2-9917-c67230252f7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:26.885772 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e302d5c-94d0-43d2-9917-c67230252f7d service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:26.885772 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e302d5c-94d0-43d2-9917-c67230252f7d service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:26.885772 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e302d5c-94d0-43d2-9917-c67230252f7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:26.892243 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e302d5c-94d0-43d2-9917-c67230252f7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:26.892930 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e302d5c-94d0-43d2-9917-c67230252f7d service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:26.904747 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 997/1969] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:26:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:26:26.955410 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f32ed6-3e71-4f58-97ae-460738ad528d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:26.957086 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f32ed6-3e71-4f58-97ae-460738ad528d service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:26.970701 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 973/1970] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:26:26 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:26:27.377874 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=85,keystone:INSERT=6,keystone:DELETE=9 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:26:27.535434 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e302d5c-94d0-43d2-9917-c67230252f7d service neutron] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:27.537582 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-1e302d5c-94d0-43d2-9917-c67230252f7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:27.553816 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e302d5c-94d0-43d2-9917-c67230252f7d service neutron] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:27.590040 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c264cbd2-bc20-4f30-a14f-7e91e00a2f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:27.590998 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c264cbd2-bc20-4f30-a14f-7e91e00a2f7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:27.591131 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c264cbd2-bc20-4f30-a14f-7e91e00a2f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:27.591420 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c264cbd2-bc20-4f30-a14f-7e91e00a2f7d admin admin] PATH_INFO: `/v3/projects/99addd712333446abbe3593b715d1e4a` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:27.592622 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c264cbd2-bc20-4f30-a14f-7e91e00a2f7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=99addd712333446abbe3593b715d1e4a)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:27.595257 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c264cbd2-bc20-4f30-a14f-7e91e00a2f7d admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:27.792312 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-c264cbd2-bc20-4f30-a14f-7e91e00a2f7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99addd712333446abbe3593b715d1e4a', 'request_id': 'req-c264cbd2-bc20-4f30-a14f-7e91e00a2f7d'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:27.798462 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-c264cbd2-bc20-4f30-a14f-7e91e00a2f7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99addd712333446abbe3593b715d1e4a', 'request_id': 'req-c264cbd2-bc20-4f30-a14f-7e91e00a2f7d'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:27.838474 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:26:27.855104 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-c264cbd2-bc20-4f30-a14f-7e91e00a2f7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99addd712333446abbe3593b715d1e4a', 'request_id': 'req-c264cbd2-bc20-4f30-a14f-7e91e00a2f7d'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:27.867696 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-c264cbd2-bc20-4f30-a14f-7e91e00a2f7d admin admin] The token cache is being invalidate because project 99addd712333446abbe3593b715d1e4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:26:27.872599 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-c264cbd2-bc20-4f30-a14f-7e91e00a2f7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:27.872599 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-c264cbd2-bc20-4f30-a14f-7e91e00a2f7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:27.872599 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 998/1971] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:26:27 2022] DELETE /identity/v3/projects/99addd712333446abbe3593b715d1e4a => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:26:27.874303 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f32ed6-3e71-4f58-97ae-460738ad528d service cinder] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:27.886408 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f32ed6-3e71-4f58-97ae-460738ad528d service cinder] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:27.961788 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0468d5ca-9c5b-49a1-96e7-d937b354fcc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:27.962441 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0468d5ca-9c5b-49a1-96e7-d937b354fcc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:27.962725 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0468d5ca-9c5b-49a1-96e7-d937b354fcc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:27.963001 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0468d5ca-9c5b-49a1-96e7-d937b354fcc4 admin admin] PATH_INFO: `/v3/users/b6959c4393be4ad1ae22133707a704a6` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:27.964208 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0468d5ca-9c5b-49a1-96e7-d937b354fcc4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6959c4393be4ad1ae22133707a704a6)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:27.966152 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0468d5ca-9c5b-49a1-96e7-d937b354fcc4 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:28.202010 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-0468d5ca-9c5b-49a1-96e7-d937b354fcc4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6959c4393be4ad1ae22133707a704a6'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:28.214694 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-0468d5ca-9c5b-49a1-96e7-d937b354fcc4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6959c4393be4ad1ae22133707a704a6'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:28.215439 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-0468d5ca-9c5b-49a1-96e7-d937b354fcc4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6959c4393be4ad1ae22133707a704a6'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:28.222404 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-0468d5ca-9c5b-49a1-96e7-d937b354fcc4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6959c4393be4ad1ae22133707a704a6'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:28.223091 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-0468d5ca-9c5b-49a1-96e7-d937b354fcc4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6959c4393be4ad1ae22133707a704a6'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:28.331004 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 974/1972] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:26:27 2022] DELETE /identity/v3/users/b6959c4393be4ad1ae22133707a704a6 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:26:28.771488 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c264cbd2-bc20-4f30-a14f-7e91e00a2f7d admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:28.773493 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-c264cbd2-bc20-4f30-a14f-7e91e00a2f7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:28.791131 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c264cbd2-bc20-4f30-a14f-7e91e00a2f7d admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:28.875254 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f40f58-c25e-4649-87b5-b61be6366b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:28.876554 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f40f58-c25e-4649-87b5-b61be6366b68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:28.876827 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f40f58-c25e-4649-87b5-b61be6366b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:28.877499 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f40f58-c25e-4649-87b5-b61be6366b68 admin admin] PATH_INFO: `/v3/projects/7d40d66b6f5d4c64bee2257b7fd35b84` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:28.879924 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f40f58-c25e-4649-87b5-b61be6366b68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d40d66b6f5d4c64bee2257b7fd35b84)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:28.881828 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f40f58-c25e-4649-87b5-b61be6366b68 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:28.989276 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-28f40f58-c25e-4649-87b5-b61be6366b68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d40d66b6f5d4c64bee2257b7fd35b84', 'request_id': 'req-28f40f58-c25e-4649-87b5-b61be6366b68'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:28.995242 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-28f40f58-c25e-4649-87b5-b61be6366b68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d40d66b6f5d4c64bee2257b7fd35b84', 'request_id': 'req-28f40f58-c25e-4649-87b5-b61be6366b68'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:29.085048 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:DELETE=1,keystone:SELECT=99 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:26:29.086030 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-28f40f58-c25e-4649-87b5-b61be6366b68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d40d66b6f5d4c64bee2257b7fd35b84', 'request_id': 'req-28f40f58-c25e-4649-87b5-b61be6366b68'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:29.106272 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-28f40f58-c25e-4649-87b5-b61be6366b68 admin admin] The token cache is being invalidate because project 7d40d66b6f5d4c64bee2257b7fd35b84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:26:29.106768 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-28f40f58-c25e-4649-87b5-b61be6366b68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:29.107762 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-28f40f58-c25e-4649-87b5-b61be6366b68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:29.110672 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 999/1973] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:26:28 2022] DELETE /identity/v3/projects/7d40d66b6f5d4c64bee2257b7fd35b84 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:26:29.116522 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac9b340-f6e5-4297-b927-ab38f1376145 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:29.117380 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac9b340-f6e5-4297-b927-ab38f1376145 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:29.117747 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac9b340-f6e5-4297-b927-ab38f1376145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:29.140117 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-fac9b340-f6e5-4297-b927-ab38f1376145 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:26:29.141667 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-fac9b340-f6e5-4297-b927-ab38f1376145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:29.209091 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 975/1974] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:26:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:26:29.217643 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33099150-48e7-4609-9586-acf5aa5663f0 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:29.218213 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33099150-48e7-4609-9586-acf5aa5663f0 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:29.218501 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33099150-48e7-4609-9586-acf5aa5663f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:29.240769 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-33099150-48e7-4609-9586-acf5aa5663f0 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:26:29.242002 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-33099150-48e7-4609-9586-acf5aa5663f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:29.250043 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1000/1975] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:26:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:26:29.254572 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac9b340-f6e5-4297-b927-ab38f1376145 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:29.265540 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac9b340-f6e5-4297-b927-ab38f1376145 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:29.270460 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6788c63-87dd-48c0-8d15-ce9e14253784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:29.272492 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6788c63-87dd-48c0-8d15-ce9e14253784 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:29.272788 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6788c63-87dd-48c0-8d15-ce9e14253784 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:29.273061 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6788c63-87dd-48c0-8d15-ce9e14253784 admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:29.273773 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6788c63-87dd-48c0-8d15-ce9e14253784 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:26:29.274106 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6788c63-87dd-48c0-8d15-ce9e14253784 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:29.275939 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6788c63-87dd-48c0-8d15-ce9e14253784 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:29.288377 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 976/1976] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Aug 4 21:26:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:26:29.292885 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33099150-48e7-4609-9586-acf5aa5663f0 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:29.302914 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33099150-48e7-4609-9586-acf5aa5663f0 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:29.307825 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb6f193-0896-4207-85ef-df57d5917a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:29.308440 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb6f193-0896-4207-85ef-df57d5917a3e admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:29.308722 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb6f193-0896-4207-85ef-df57d5917a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:29.308995 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb6f193-0896-4207-85ef-df57d5917a3e admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:29.309676 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb6f193-0896-4207-85ef-df57d5917a3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:29.311705 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb6f193-0896-4207-85ef-df57d5917a3e admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:29.445053 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1001/1977] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:26:29 2022] POST /identity/v3/projects => generated 377 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:26:29.449045 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6788c63-87dd-48c0-8d15-ce9e14253784 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:29.458917 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6788c63-87dd-48c0-8d15-ce9e14253784 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:29.514276 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:26:29.522152 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e3d81f-83db-492a-b297-a2663a6dac6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:29.522918 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e3d81f-83db-492a-b297-a2663a6dac6e admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:29.523259 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e3d81f-83db-492a-b297-a2663a6dac6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:29.523639 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e3d81f-83db-492a-b297-a2663a6dac6e admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:29.524494 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e3d81f-83db-492a-b297-a2663a6dac6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:29.527071 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e3d81f-83db-492a-b297-a2663a6dac6e admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:29.700063 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 977/1978] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:26:29 2022] POST /identity/v3/users => generated 289 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:26:29.700197 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb6f193-0896-4207-85ef-df57d5917a3e admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:29.709034 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb6f193-0896-4207-85ef-df57d5917a3e admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:29.714018 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f2e106-021a-4fb5-9f48-224c6fa1b78e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:29.714675 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f2e106-021a-4fb5-9f48-224c6fa1b78e admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:29.714988 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f2e106-021a-4fb5-9f48-224c6fa1b78e admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:29.715270 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f2e106-021a-4fb5-9f48-224c6fa1b78e admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:29.716006 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f2e106-021a-4fb5-9f48-224c6fa1b78e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:29.717640 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f2e106-021a-4fb5-9f48-224c6fa1b78e admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:29.728415 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1002/1979] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:26:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:26:29.738119 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e3d81f-83db-492a-b297-a2663a6dac6e admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:29.750736 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e3d81f-83db-492a-b297-a2663a6dac6e admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:29.757774 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6455fe6-954a-4316-b691-d6fea01b752c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:29.760963 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6455fe6-954a-4316-b691-d6fea01b752c admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:29.760963 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6455fe6-954a-4316-b691-d6fea01b752c admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:29.760963 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6455fe6-954a-4316-b691-d6fea01b752c admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:29.760963 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6455fe6-954a-4316-b691-d6fea01b752c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:29.763808 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6455fe6-954a-4316-b691-d6fea01b752c admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:29.782781 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 978/1980] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:26:29 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:26:29.787843 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f2e106-021a-4fb5-9f48-224c6fa1b78e admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:29.802420 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f2e106-021a-4fb5-9f48-224c6fa1b78e admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:29.807358 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d62fa6-8d26-4126-8f55-f0b3e88085a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:29.808002 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d62fa6-8d26-4126-8f55-f0b3e88085a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:29.808288 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d62fa6-8d26-4126-8f55-f0b3e88085a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:29.808570 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d62fa6-8d26-4126-8f55-f0b3e88085a3 admin admin] PATH_INFO: `/v3/projects/eab4594071a64f71aa3f31735de103d3/users/948c7bacbadc4676ba75b1d29e6c0bd1/roles/8feebc60363c4feb870718b1721e5324` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:29.829488 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d62fa6-8d26-4126-8f55-f0b3e88085a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eab4594071a64f71aa3f31735de103d3, user_id=948c7bacbadc4676ba75b1d29e6c0bd1, role_id=8feebc60363c4feb870718b1721e5324)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:29.834740 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d62fa6-8d26-4126-8f55-f0b3e88085a3 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:29.985444 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1003/1981] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:26:29 2022] PUT /identity/v3/projects/eab4594071a64f71aa3f31735de103d3/users/948c7bacbadc4676ba75b1d29e6c0bd1/roles/8feebc60363c4feb870718b1721e5324 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:26:29.997089 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e97c6d-d3bb-40a3-b684-36764d4166ad None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:29.997526 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e97c6d-d3bb-40a3-b684-36764d4166ad None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:29.997960 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e97c6d-d3bb-40a3-b684-36764d4166ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:30.030444 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-94e97c6d-d3bb-40a3-b684-36764d4166ad None None] MFA Rules not processed for user `948c7bacbadc4676ba75b1d29e6c0bd1`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:26:30.032279 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-94e97c6d-d3bb-40a3-b684-36764d4166ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:30.116252 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 979/1982] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:26:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:26:30.128966 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d62fa6-8d26-4126-8f55-f0b3e88085a3 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:30.130786 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-b0d62fa6-8d26-4126-8f55-f0b3e88085a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:30.227854 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d62fa6-8d26-4126-8f55-f0b3e88085a3 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:30.235726 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2c990b-c4b1-4fee-bf1f-5ae750f676c6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6885c7d929f4f8e96d3890724c3b53f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:30.236440 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2c990b-c4b1-4fee-bf1f-5ae750f676c6 service glance] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:30.236995 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2c990b-c4b1-4fee-bf1f-5ae750f676c6 service glance] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:30.237109 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2c990b-c4b1-4fee-bf1f-5ae750f676c6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:30.252772 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2c990b-c4b1-4fee-bf1f-5ae750f676c6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:30.254294 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2c990b-c4b1-4fee-bf1f-5ae750f676c6 service glance] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:30.264606 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1004/1983] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:26:30 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:26:30.370397 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:26:30.873330 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:26:31.654425 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e97c6d-d3bb-40a3-b684-36764d4166ad None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:31.659474 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-94e97c6d-d3bb-40a3-b684-36764d4166ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:31.735921 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e97c6d-d3bb-40a3-b684-36764d4166ad None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:31.740754 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562f478f-57c8-45ea-8047-2be77b1160dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:31.741418 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f478f-57c8-45ea-8047-2be77b1160dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:31.741703 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f478f-57c8-45ea-8047-2be77b1160dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:31.741981 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f478f-57c8-45ea-8047-2be77b1160dc admin admin] PATH_INFO: `/v3/users/7c67b93ddfef492a9c834a4f6dc6308b` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:31.743178 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562f478f-57c8-45ea-8047-2be77b1160dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c67b93ddfef492a9c834a4f6dc6308b)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:31.745181 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562f478f-57c8-45ea-8047-2be77b1160dc admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:32.003344 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-562f478f-57c8-45ea-8047-2be77b1160dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c67b93ddfef492a9c834a4f6dc6308b'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:32.003344 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:26:32.010503 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-562f478f-57c8-45ea-8047-2be77b1160dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c67b93ddfef492a9c834a4f6dc6308b'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:32.011287 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-562f478f-57c8-45ea-8047-2be77b1160dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c67b93ddfef492a9c834a4f6dc6308b'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:32.019723 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-562f478f-57c8-45ea-8047-2be77b1160dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c67b93ddfef492a9c834a4f6dc6308b'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:32.020554 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-562f478f-57c8-45ea-8047-2be77b1160dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c67b93ddfef492a9c834a4f6dc6308b'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:32.125040 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 980/1984] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:26:31 2022] DELETE /identity/v3/users/7c67b93ddfef492a9c834a4f6dc6308b => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:26:32.130078 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b78d49-069d-4774-be21-4779a0c535d8 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:32.132021 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b78d49-069d-4774-be21-4779a0c535d8 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:32.132021 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b78d49-069d-4774-be21-4779a0c535d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:32.152724 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-f1b78d49-069d-4774-be21-4779a0c535d8 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:26:32.153967 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-f1b78d49-069d-4774-be21-4779a0c535d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:32.200259 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:26:32.227423 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1005/1985] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:26:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:26:32.242306 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562f478f-57c8-45ea-8047-2be77b1160dc admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:32.244054 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-562f478f-57c8-45ea-8047-2be77b1160dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:32.327019 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562f478f-57c8-45ea-8047-2be77b1160dc admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:32.332674 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf178a6-5974-4371-8d5b-1f0767b02f8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:32.333459 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf178a6-5974-4371-8d5b-1f0767b02f8a service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:32.333808 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf178a6-5974-4371-8d5b-1f0767b02f8a service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:32.334113 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf178a6-5974-4371-8d5b-1f0767b02f8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:32.352030 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf178a6-5974-4371-8d5b-1f0767b02f8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:32.353566 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf178a6-5974-4371-8d5b-1f0767b02f8a service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:32.365910 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 981/1986] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:26:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:26:33.301102 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b78d49-069d-4774-be21-4779a0c535d8 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:33.301102 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-f1b78d49-069d-4774-be21-4779a0c535d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:33.319298 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b78d49-069d-4774-be21-4779a0c535d8 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:33.340738 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dd3a54-6b1f-4853-9a76-4c1a5faa3aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:33.340738 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dd3a54-6b1f-4853-9a76-4c1a5faa3aa3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:33.340738 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dd3a54-6b1f-4853-9a76-4c1a5faa3aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:33.340738 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dd3a54-6b1f-4853-9a76-4c1a5faa3aa3 admin admin] PATH_INFO: `/v3/projects/c6d2209282e9415688759b7067c420e3` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:33.341092 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dd3a54-6b1f-4853-9a76-4c1a5faa3aa3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6d2209282e9415688759b7067c420e3)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:33.341092 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dd3a54-6b1f-4853-9a76-4c1a5faa3aa3 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:33.474251 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-b0dd3a54-6b1f-4853-9a76-4c1a5faa3aa3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6d2209282e9415688759b7067c420e3', 'request_id': 'req-b0dd3a54-6b1f-4853-9a76-4c1a5faa3aa3'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:33.480418 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-b0dd3a54-6b1f-4853-9a76-4c1a5faa3aa3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6d2209282e9415688759b7067c420e3', 'request_id': 'req-b0dd3a54-6b1f-4853-9a76-4c1a5faa3aa3'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:33.643681 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-b0dd3a54-6b1f-4853-9a76-4c1a5faa3aa3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6d2209282e9415688759b7067c420e3', 'request_id': 'req-b0dd3a54-6b1f-4853-9a76-4c1a5faa3aa3'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:33.664924 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-b0dd3a54-6b1f-4853-9a76-4c1a5faa3aa3 admin admin] The token cache is being invalidate because project c6d2209282e9415688759b7067c420e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:26:33.665221 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-b0dd3a54-6b1f-4853-9a76-4c1a5faa3aa3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:33.665903 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-b0dd3a54-6b1f-4853-9a76-4c1a5faa3aa3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:33.667791 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1006/1987] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:26:33 2022] DELETE /identity/v3/projects/c6d2209282e9415688759b7067c420e3 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:26:33.674010 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcb66ce-becc-40ff-94ea-8bd5b7830f8b None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:33.674010 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcb66ce-becc-40ff-94ea-8bd5b7830f8b None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:33.676233 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcb66ce-becc-40ff-94ea-8bd5b7830f8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:33.704157 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-6bcb66ce-becc-40ff-94ea-8bd5b7830f8b None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:26:33.707880 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-6bcb66ce-becc-40ff-94ea-8bd5b7830f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:33.773040 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 982/1988] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:26:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:26:33.782209 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412346c7-9814-490a-8d9e-27a1203e2437 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:33.782500 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412346c7-9814-490a-8d9e-27a1203e2437 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:33.782778 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412346c7-9814-490a-8d9e-27a1203e2437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:33.808024 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-412346c7-9814-490a-8d9e-27a1203e2437 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:26:33.809852 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-412346c7-9814-490a-8d9e-27a1203e2437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:33.820236 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1007/1989] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:26:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:26:33.826434 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcb66ce-becc-40ff-94ea-8bd5b7830f8b None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:33.842380 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcb66ce-becc-40ff-94ea-8bd5b7830f8b None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:33.848030 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587eb0fa-cd3e-41e1-9ed3-d5b7d0ab0baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:33.851640 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587eb0fa-cd3e-41e1-9ed3-d5b7d0ab0baf admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:33.851640 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587eb0fa-cd3e-41e1-9ed3-d5b7d0ab0baf admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:33.851640 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587eb0fa-cd3e-41e1-9ed3-d5b7d0ab0baf admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:33.851640 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587eb0fa-cd3e-41e1-9ed3-d5b7d0ab0baf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:26:33.851640 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587eb0fa-cd3e-41e1-9ed3-d5b7d0ab0baf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:33.852070 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587eb0fa-cd3e-41e1-9ed3-d5b7d0ab0baf admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:33.866117 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 983/1990] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Aug 4 21:26:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:26:33.870029 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412346c7-9814-490a-8d9e-27a1203e2437 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:33.880044 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412346c7-9814-490a-8d9e-27a1203e2437 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:33.884773 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f484918-acb9-4d22-9dca-4e2e984cce1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:33.885393 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f484918-acb9-4d22-9dca-4e2e984cce1e admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:33.885674 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f484918-acb9-4d22-9dca-4e2e984cce1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:33.885962 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f484918-acb9-4d22-9dca-4e2e984cce1e admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:33.886633 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f484918-acb9-4d22-9dca-4e2e984cce1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:33.888595 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f484918-acb9-4d22-9dca-4e2e984cce1e admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:33.990007 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1008/1991] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:26:33 2022] POST /identity/v3/projects => generated 399 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:26:33.996049 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587eb0fa-cd3e-41e1-9ed3-d5b7d0ab0baf admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:34.007067 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587eb0fa-cd3e-41e1-9ed3-d5b7d0ab0baf admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:34.090063 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db12f457-64fc-4921-afe6-4b57ee4ab475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:34.091215 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db12f457-64fc-4921-afe6-4b57ee4ab475 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:34.092946 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db12f457-64fc-4921-afe6-4b57ee4ab475 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:34.092946 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db12f457-64fc-4921-afe6-4b57ee4ab475 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:34.093372 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db12f457-64fc-4921-afe6-4b57ee4ab475 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:34.095712 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db12f457-64fc-4921-afe6-4b57ee4ab475 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:34.135247 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:26:34.238723 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 984/1992] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:26:33 2022] POST /identity/v3/users => generated 300 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:26:34.243273 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f484918-acb9-4d22-9dca-4e2e984cce1e admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:34.262176 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f484918-acb9-4d22-9dca-4e2e984cce1e admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:34.269794 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504231a4-5923-4e8f-9509-3b0f3530331d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:34.270698 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504231a4-5923-4e8f-9509-3b0f3530331d admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:34.271249 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504231a4-5923-4e8f-9509-3b0f3530331d admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:34.271751 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504231a4-5923-4e8f-9509-3b0f3530331d admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:34.272965 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504231a4-5923-4e8f-9509-3b0f3530331d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:34.275852 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504231a4-5923-4e8f-9509-3b0f3530331d admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:34.293478 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1009/1993] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:26:34 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:26:34.299725 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db12f457-64fc-4921-afe6-4b57ee4ab475 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:34.310225 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db12f457-64fc-4921-afe6-4b57ee4ab475 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:34.315766 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8957dc7-4277-487a-930d-fb8f676a735b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:34.316533 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8957dc7-4277-487a-930d-fb8f676a735b admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:34.316820 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8957dc7-4277-487a-930d-fb8f676a735b admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:34.317179 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8957dc7-4277-487a-930d-fb8f676a735b admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:34.318822 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8957dc7-4277-487a-930d-fb8f676a735b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:34.319670 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8957dc7-4277-487a-930d-fb8f676a735b admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:34.332575 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 985/1994] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:26:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:26:34.337254 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504231a4-5923-4e8f-9509-3b0f3530331d admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:34.351412 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504231a4-5923-4e8f-9509-3b0f3530331d admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:34.355926 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a14823-ceaa-4fa3-b4ab-ec35162eae15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:34.356818 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a14823-ceaa-4fa3-b4ab-ec35162eae15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:34.357149 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a14823-ceaa-4fa3-b4ab-ec35162eae15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:34.357568 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a14823-ceaa-4fa3-b4ab-ec35162eae15 admin admin] PATH_INFO: `/v3/projects/354c3d02c2374126be9d94268abdbed0/users/62d2c78b310e4b43a5a84d76f078b16a/roles/8feebc60363c4feb870718b1721e5324` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:34.385770 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a14823-ceaa-4fa3-b4ab-ec35162eae15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=354c3d02c2374126be9d94268abdbed0, user_id=62d2c78b310e4b43a5a84d76f078b16a, role_id=8feebc60363c4feb870718b1721e5324)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:34.388491 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a14823-ceaa-4fa3-b4ab-ec35162eae15 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:34.492518 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1010/1995] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:26:34 2022] PUT /identity/v3/projects/354c3d02c2374126be9d94268abdbed0/users/62d2c78b310e4b43a5a84d76f078b16a/roles/8feebc60363c4feb870718b1721e5324 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:26:34.504028 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f48e439-029c-46a1-a3c8-d3500c864a64 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:34.504313 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f48e439-029c-46a1-a3c8-d3500c864a64 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:34.504614 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f48e439-029c-46a1-a3c8-d3500c864a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:34.546345 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-2f48e439-029c-46a1-a3c8-d3500c864a64 None None] MFA Rules not processed for user `62d2c78b310e4b43a5a84d76f078b16a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:26:34.548156 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-2f48e439-029c-46a1-a3c8-d3500c864a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:34.621722 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 986/1996] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:26:34 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:26:34.634095 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a14823-ceaa-4fa3-b4ab-ec35162eae15 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:34.636144 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-17a14823-ceaa-4fa3-b4ab-ec35162eae15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:34.700100 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a14823-ceaa-4fa3-b4ab-ec35162eae15 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:34.704346 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e8d338-59f8-4f3b-9f4c-8e139aaac2d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e31f2728056b460fa474478eb85ca1d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:34.705073 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e8d338-59f8-4f3b-9f4c-8e139aaac2d5 service cinder] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:34.705548 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e8d338-59f8-4f3b-9f4c-8e139aaac2d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:34.705826 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e8d338-59f8-4f3b-9f4c-8e139aaac2d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:34.716768 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e8d338-59f8-4f3b-9f4c-8e139aaac2d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:34.725405 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e8d338-59f8-4f3b-9f4c-8e139aaac2d5 service cinder] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:34.728988 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1011/1997] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:26:34 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:26:35.444069 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:26:36.090427 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:26:36.381294 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:26:36.829358 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:26:37.291671 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:26:37.772062 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:26:38.136674 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:26:38.331234 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:26:38.861997 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:26:38.901182 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:26:38.987127 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:26:39.062742 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:26:39.083870 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:26:39.193272 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:26:39.234023 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:26:44.243569 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f48e439-029c-46a1-a3c8-d3500c864a64 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:44.278862 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:26:44.307748 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f48e439-029c-46a1-a3c8-d3500c864a64 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:44.312619 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5edf5c-0e22-491d-ac2d-40c068b35365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:44.313271 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5edf5c-0e22-491d-ac2d-40c068b35365 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:44.314578 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5edf5c-0e22-491d-ac2d-40c068b35365 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:44.314856 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5edf5c-0e22-491d-ac2d-40c068b35365 admin admin] PATH_INFO: `/v3/users/62d2c78b310e4b43a5a84d76f078b16a` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:44.316047 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5edf5c-0e22-491d-ac2d-40c068b35365 admin admin] RBAC: Authorizing `identity:delete_user(user_id=62d2c78b310e4b43a5a84d76f078b16a)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:44.318008 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5edf5c-0e22-491d-ac2d-40c068b35365 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:44.566444 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-2a5edf5c-0e22-491d-ac2d-40c068b35365 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62d2c78b310e4b43a5a84d76f078b16a'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:44.578796 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-2a5edf5c-0e22-491d-ac2d-40c068b35365 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62d2c78b310e4b43a5a84d76f078b16a'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:44.579514 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-2a5edf5c-0e22-491d-ac2d-40c068b35365 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62d2c78b310e4b43a5a84d76f078b16a'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:44.586427 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-2a5edf5c-0e22-491d-ac2d-40c068b35365 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62d2c78b310e4b43a5a84d76f078b16a'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:44.587107 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-2a5edf5c-0e22-491d-ac2d-40c068b35365 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62d2c78b310e4b43a5a84d76f078b16a'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:44.707609 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 987/1998] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:26:44 2022] DELETE /identity/v3/users/62d2c78b310e4b43a5a84d76f078b16a => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:26:44.712101 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58366a2-e729-4dc5-b745-bb64ca850770 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:44.712541 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58366a2-e729-4dc5-b745-bb64ca850770 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:44.712652 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58366a2-e729-4dc5-b745-bb64ca850770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:44.733129 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-c58366a2-e729-4dc5-b745-bb64ca850770 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:26:44.734347 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-c58366a2-e729-4dc5-b745-bb64ca850770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:44.778142 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:26:44.822373 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1012/1999] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:26:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:26:44.841781 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5edf5c-0e22-491d-ac2d-40c068b35365 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:44.843260 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-2a5edf5c-0e22-491d-ac2d-40c068b35365 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:44.909753 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5edf5c-0e22-491d-ac2d-40c068b35365 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:44.913790 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204a6ede-70fe-4b89-89d4-ca8f10a8094b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:44.914475 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204a6ede-70fe-4b89-89d4-ca8f10a8094b service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:44.914770 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204a6ede-70fe-4b89-89d4-ca8f10a8094b service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:44.915049 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204a6ede-70fe-4b89-89d4-ca8f10a8094b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:44.930491 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204a6ede-70fe-4b89-89d4-ca8f10a8094b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:44.932314 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204a6ede-70fe-4b89-89d4-ca8f10a8094b service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:44.949243 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 988/2000] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:26:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:26:45.550861 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58366a2-e729-4dc5-b745-bb64ca850770 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:45.552638 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-c58366a2-e729-4dc5-b745-bb64ca850770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:45.582437 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58366a2-e729-4dc5-b745-bb64ca850770 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:45.590953 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f725210-fb04-4b2c-9606-9421c25deb55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:45.607224 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f725210-fb04-4b2c-9606-9421c25deb55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:45.607224 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f725210-fb04-4b2c-9606-9421c25deb55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:45.607224 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f725210-fb04-4b2c-9606-9421c25deb55 admin admin] PATH_INFO: `/v3/projects/354c3d02c2374126be9d94268abdbed0` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:45.607224 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f725210-fb04-4b2c-9606-9421c25deb55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=354c3d02c2374126be9d94268abdbed0)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:45.607224 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f725210-fb04-4b2c-9606-9421c25deb55 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:45.829155 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-9f725210-fb04-4b2c-9606-9421c25deb55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '354c3d02c2374126be9d94268abdbed0', 'request_id': 'req-9f725210-fb04-4b2c-9606-9421c25deb55'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:45.837264 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-9f725210-fb04-4b2c-9606-9421c25deb55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '354c3d02c2374126be9d94268abdbed0', 'request_id': 'req-9f725210-fb04-4b2c-9606-9421c25deb55'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:45.900986 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-9f725210-fb04-4b2c-9606-9421c25deb55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '354c3d02c2374126be9d94268abdbed0', 'request_id': 'req-9f725210-fb04-4b2c-9606-9421c25deb55'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:45.914609 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-9f725210-fb04-4b2c-9606-9421c25deb55 admin admin] The token cache is being invalidate because project 354c3d02c2374126be9d94268abdbed0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:26:45.915149 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-9f725210-fb04-4b2c-9606-9421c25deb55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:45.916108 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-9f725210-fb04-4b2c-9606-9421c25deb55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:45.918641 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1013/2001] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:26:45 2022] DELETE /identity/v3/projects/354c3d02c2374126be9d94268abdbed0 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:26:48.130984 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204a6ede-70fe-4b89-89d4-ca8f10a8094b service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:48.132508 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-204a6ede-70fe-4b89-89d4-ca8f10a8094b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:48.150577 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204a6ede-70fe-4b89-89d4-ca8f10a8094b service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:48.220938 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:26:48.229433 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f490d76-e44f-4cb7-833a-85a586ea26cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:48.230260 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f490d76-e44f-4cb7-833a-85a586ea26cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:48.230612 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f490d76-e44f-4cb7-833a-85a586ea26cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:48.237292 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f490d76-e44f-4cb7-833a-85a586ea26cc admin admin] PATH_INFO: `/v3/users/5296a0af07f14a7e903f2fba21d32276` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:48.237292 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f490d76-e44f-4cb7-833a-85a586ea26cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=5296a0af07f14a7e903f2fba21d32276)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:48.237292 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f490d76-e44f-4cb7-833a-85a586ea26cc admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:48.429845 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-1f490d76-e44f-4cb7-833a-85a586ea26cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5296a0af07f14a7e903f2fba21d32276'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:48.446908 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-1f490d76-e44f-4cb7-833a-85a586ea26cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5296a0af07f14a7e903f2fba21d32276'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:48.447642 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-1f490d76-e44f-4cb7-833a-85a586ea26cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5296a0af07f14a7e903f2fba21d32276'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:48.456135 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-1f490d76-e44f-4cb7-833a-85a586ea26cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5296a0af07f14a7e903f2fba21d32276'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:48.456849 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-1f490d76-e44f-4cb7-833a-85a586ea26cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5296a0af07f14a7e903f2fba21d32276'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:48.552617 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 989/2002] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:26:48 2022] DELETE /identity/v3/users/5296a0af07f14a7e903f2fba21d32276 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:26:48.818833 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f725210-fb04-4b2c-9606-9421c25deb55 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:48.818833 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-9f725210-fb04-4b2c-9606-9421c25deb55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:48.831605 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f725210-fb04-4b2c-9606-9421c25deb55 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:48.894742 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1c99b5-2c6e-4af0-85e8-12955686afc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:48.895440 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1c99b5-2c6e-4af0-85e8-12955686afc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:48.895729 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1c99b5-2c6e-4af0-85e8-12955686afc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:48.896006 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1c99b5-2c6e-4af0-85e8-12955686afc2 admin admin] PATH_INFO: `/v3/projects/46d62342f37d4d5aa77d4a42fb3cda65` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:48.897175 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1c99b5-2c6e-4af0-85e8-12955686afc2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46d62342f37d4d5aa77d4a42fb3cda65)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:48.900917 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1c99b5-2c6e-4af0-85e8-12955686afc2 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:49.042561 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-8d1c99b5-2c6e-4af0-85e8-12955686afc2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46d62342f37d4d5aa77d4a42fb3cda65', 'request_id': 'req-8d1c99b5-2c6e-4af0-85e8-12955686afc2'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:49.054591 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-8d1c99b5-2c6e-4af0-85e8-12955686afc2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46d62342f37d4d5aa77d4a42fb3cda65', 'request_id': 'req-8d1c99b5-2c6e-4af0-85e8-12955686afc2'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:49.057913 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:26:49.161976 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-8d1c99b5-2c6e-4af0-85e8-12955686afc2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46d62342f37d4d5aa77d4a42fb3cda65', 'request_id': 'req-8d1c99b5-2c6e-4af0-85e8-12955686afc2'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:49.182818 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-8d1c99b5-2c6e-4af0-85e8-12955686afc2 admin admin] The token cache is being invalidate because project 46d62342f37d4d5aa77d4a42fb3cda65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:26:49.182818 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-8d1c99b5-2c6e-4af0-85e8-12955686afc2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:49.182818 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-8d1c99b5-2c6e-4af0-85e8-12955686afc2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:49.185190 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1014/2003] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:26:48 2022] DELETE /identity/v3/projects/46d62342f37d4d5aa77d4a42fb3cda65 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:26:49.188762 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f490d76-e44f-4cb7-833a-85a586ea26cc admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:49.190529 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-1f490d76-e44f-4cb7-833a-85a586ea26cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:49.203552 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f490d76-e44f-4cb7-833a-85a586ea26cc admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:49.287917 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886be01f-3f27-4c42-a718-48a1e6e22e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:49.288873 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886be01f-3f27-4c42-a718-48a1e6e22e92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:49.289315 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886be01f-3f27-4c42-a718-48a1e6e22e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:49.289759 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886be01f-3f27-4c42-a718-48a1e6e22e92 admin admin] PATH_INFO: `/v3/users/b522296f29544acf9377a7b52314a6b8` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:49.291525 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886be01f-3f27-4c42-a718-48a1e6e22e92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b522296f29544acf9377a7b52314a6b8)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:49.293828 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886be01f-3f27-4c42-a718-48a1e6e22e92 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:49.503940 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-886be01f-3f27-4c42-a718-48a1e6e22e92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b522296f29544acf9377a7b52314a6b8'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:49.536440 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-886be01f-3f27-4c42-a718-48a1e6e22e92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b522296f29544acf9377a7b52314a6b8'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:49.537306 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-886be01f-3f27-4c42-a718-48a1e6e22e92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b522296f29544acf9377a7b52314a6b8'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:49.546380 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-886be01f-3f27-4c42-a718-48a1e6e22e92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b522296f29544acf9377a7b52314a6b8'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:49.547689 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-886be01f-3f27-4c42-a718-48a1e6e22e92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b522296f29544acf9377a7b52314a6b8'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:49.551118 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:26:49.677605 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 990/2004] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:26:49 2022] DELETE /identity/v3/users/b522296f29544acf9377a7b52314a6b8 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:26:49.768826 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8a56d5-4b98-4e69-99d9-3a1f28d7fb82 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:49.769210 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8a56d5-4b98-4e69-99d9-3a1f28d7fb82 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:49.769535 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8a56d5-4b98-4e69-99d9-3a1f28d7fb82 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:49.773423 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1015/2005] 10.222.0.25 () {54 vars in 921 bytes} [Thu Aug 4 21:26:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:26:49.791834 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be179ec-4d2e-4637-9651-c3c4737f39b8 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:49.792658 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be179ec-4d2e-4637-9651-c3c4737f39b8 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:49.792658 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be179ec-4d2e-4637-9651-c3c4737f39b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:49.828785 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-4be179ec-4d2e-4637-9651-c3c4737f39b8 None None] MFA Rules not processed for user `67d44a0098ab4c36b14b36e9e59ae75c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:26:49.832922 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-4be179ec-4d2e-4637-9651-c3c4737f39b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:49.936867 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 991/2006] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Aug 4 21:26:49 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:26:49.948616 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8a56d5-4b98-4e69-99d9-3a1f28d7fb82 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:49.950785 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-fe8a56d5-4b98-4e69-99d9-3a1f28d7fb82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:49.963066 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be179ec-4d2e-4637-9651-c3c4737f39b8 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:49.964714 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-4be179ec-4d2e-4637-9651-c3c4737f39b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:49.991282 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8a56d5-4b98-4e69-99d9-3a1f28d7fb82 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:49.997841 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f2b442-a2c1-4bef-97ce-e7c6d2f5ac08 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67d44a0098ab4c36b14b36e9e59ae75c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:49.998735 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f2b442-a2c1-4bef-97ce-e7c6d2f5ac08 service placement] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:49.999134 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f2b442-a2c1-4bef-97ce-e7c6d2f5ac08 service placement] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:49.999557 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f2b442-a2c1-4bef-97ce-e7c6d2f5ac08 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:50.013130 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f2b442-a2c1-4bef-97ce-e7c6d2f5ac08 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:50.013130 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f2b442-a2c1-4bef-97ce-e7c6d2f5ac08 service placement] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:50.024007 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1016/2007] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:26:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:26:50.024827 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f2b442-a2c1-4bef-97ce-e7c6d2f5ac08 service placement] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:50.026374 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-d9f2b442-a2c1-4bef-97ce-e7c6d2f5ac08 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:50.040751 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f2b442-a2c1-4bef-97ce-e7c6d2f5ac08 service placement] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:50.041564 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be179ec-4d2e-4637-9651-c3c4737f39b8 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:50.046033 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a256f3-7875-450f-a340-dd04b7cedec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:50.046746 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a256f3-7875-450f-a340-dd04b7cedec9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:50.047123 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a256f3-7875-450f-a340-dd04b7cedec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:50.047659 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a256f3-7875-450f-a340-dd04b7cedec9 admin admin] PATH_INFO: `/v3/users/966b1056487d435ca83b6a9b15b0ecea` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:50.048930 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a256f3-7875-450f-a340-dd04b7cedec9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=966b1056487d435ca83b6a9b15b0ecea)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:50.050135 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fda8bea-6afd-4a03-9cf8-f2843c4abe15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:50.051008 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a256f3-7875-450f-a340-dd04b7cedec9 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:50.051407 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fda8bea-6afd-4a03-9cf8-f2843c4abe15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:50.051763 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fda8bea-6afd-4a03-9cf8-f2843c4abe15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:50.052107 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fda8bea-6afd-4a03-9cf8-f2843c4abe15 admin admin] PATH_INFO: `/v3/projects/9b566d7b2e6b49178a9afd6a9fc0e86a` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:50.053494 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fda8bea-6afd-4a03-9cf8-f2843c4abe15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b566d7b2e6b49178a9afd6a9fc0e86a)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:50.056134 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fda8bea-6afd-4a03-9cf8-f2843c4abe15 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:50.230775 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-3fda8bea-6afd-4a03-9cf8-f2843c4abe15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b566d7b2e6b49178a9afd6a9fc0e86a', 'request_id': 'req-3fda8bea-6afd-4a03-9cf8-f2843c4abe15'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:50.231426 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-c3a256f3-7875-450f-a340-dd04b7cedec9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '966b1056487d435ca83b6a9b15b0ecea'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:50.237269 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-3fda8bea-6afd-4a03-9cf8-f2843c4abe15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b566d7b2e6b49178a9afd6a9fc0e86a', 'request_id': 'req-3fda8bea-6afd-4a03-9cf8-f2843c4abe15'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:50.243236 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-c3a256f3-7875-450f-a340-dd04b7cedec9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '966b1056487d435ca83b6a9b15b0ecea'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:50.243994 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-c3a256f3-7875-450f-a340-dd04b7cedec9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '966b1056487d435ca83b6a9b15b0ecea'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:50.244644 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-c3a256f3-7875-450f-a340-dd04b7cedec9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '966b1056487d435ca83b6a9b15b0ecea'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:50.312285 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-3fda8bea-6afd-4a03-9cf8-f2843c4abe15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b566d7b2e6b49178a9afd6a9fc0e86a', 'request_id': 'req-3fda8bea-6afd-4a03-9cf8-f2843c4abe15'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:50.327383 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-3fda8bea-6afd-4a03-9cf8-f2843c4abe15 admin admin] The token cache is being invalidate because project 9b566d7b2e6b49178a9afd6a9fc0e86a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:26:50.327725 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-3fda8bea-6afd-4a03-9cf8-f2843c4abe15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:50.328388 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-3fda8bea-6afd-4a03-9cf8-f2843c4abe15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:50.329947 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 992/2008] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:26:49 2022] DELETE /identity/v3/projects/9b566d7b2e6b49178a9afd6a9fc0e86a => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:26:50.334625 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d6ac6f-1c7b-404d-926d-6f95d662c611 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:50.334914 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d6ac6f-1c7b-404d-926d-6f95d662c611 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:50.335204 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d6ac6f-1c7b-404d-926d-6f95d662c611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:50.354725 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-67d6ac6f-1c7b-404d-926d-6f95d662c611 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:26:50.356012 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-67d6ac6f-1c7b-404d-926d-6f95d662c611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:50.380714 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-c3a256f3-7875-450f-a340-dd04b7cedec9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '966b1056487d435ca83b6a9b15b0ecea'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:50.394179 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1017/2009] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:26:50 2022] DELETE /identity/v3/users/966b1056487d435ca83b6a9b15b0ecea => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:26:50.409938 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:26:50.424058 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 993/2010] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:26:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:26:50.425708 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00319790-2932-4209-a228-4da01c9dd29f None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:50.426000 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00319790-2932-4209-a228-4da01c9dd29f None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:50.426277 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00319790-2932-4209-a228-4da01c9dd29f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:50.449146 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-00319790-2932-4209-a228-4da01c9dd29f None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:26:50.450369 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-00319790-2932-4209-a228-4da01c9dd29f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:50.527630 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1018/2011] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:26:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:26:50.528642 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d6ac6f-1c7b-404d-926d-6f95d662c611 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:50.539031 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d6ac6f-1c7b-404d-926d-6f95d662c611 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:50.544235 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98b8367-421e-480a-9b31-1bc9f3c83108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:50.544603 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98b8367-421e-480a-9b31-1bc9f3c83108 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:50.544881 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98b8367-421e-480a-9b31-1bc9f3c83108 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:50.545154 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98b8367-421e-480a-9b31-1bc9f3c83108 admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:50.545851 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98b8367-421e-480a-9b31-1bc9f3c83108 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:26:50.546153 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98b8367-421e-480a-9b31-1bc9f3c83108 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:50.548524 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98b8367-421e-480a-9b31-1bc9f3c83108 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:50.560651 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 994/2012] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Aug 4 21:26:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:26:50.566302 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00319790-2932-4209-a228-4da01c9dd29f None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:50.577603 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00319790-2932-4209-a228-4da01c9dd29f None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:50.583232 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd77740e-c985-4089-927f-31202babcb77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:50.584111 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd77740e-c985-4089-927f-31202babcb77 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:50.585140 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd77740e-c985-4089-927f-31202babcb77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:50.585140 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd77740e-c985-4089-927f-31202babcb77 admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:50.585887 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd77740e-c985-4089-927f-31202babcb77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:50.588119 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd77740e-c985-4089-927f-31202babcb77 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:50.710920 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98b8367-421e-480a-9b31-1bc9f3c83108 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:50.711931 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1019/2013] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:26:50 2022] POST /identity/v3/projects => generated 377 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:26:50.716381 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd77740e-c985-4089-927f-31202babcb77 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:50.716545 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-c98b8367-421e-480a-9b31-1bc9f3c83108 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:50.742361 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98b8367-421e-480a-9b31-1bc9f3c83108 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:50.753240 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:26:50.789713 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd77740e-c985-4089-927f-31202babcb77 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:50.790807 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3227525-6581-4a0b-8427-8fe11bcb2d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:50.791493 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3227525-6581-4a0b-8427-8fe11bcb2d96 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:50.791773 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3227525-6581-4a0b-8427-8fe11bcb2d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:50.792043 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3227525-6581-4a0b-8427-8fe11bcb2d96 admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:50.792709 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3227525-6581-4a0b-8427-8fe11bcb2d96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:50.794662 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3227525-6581-4a0b-8427-8fe11bcb2d96 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:50.802257 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c33ebb3-4b2c-46e2-8fc0-d91792a46604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:50.802910 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c33ebb3-4b2c-46e2-8fc0-d91792a46604 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:50.803192 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c33ebb3-4b2c-46e2-8fc0-d91792a46604 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:50.803496 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c33ebb3-4b2c-46e2-8fc0-d91792a46604 admin admin] PATH_INFO: `/v3/projects/33e98e2129fe47bd8f81e1185d6a33a8` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:50.804647 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c33ebb3-4b2c-46e2-8fc0-d91792a46604 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33e98e2129fe47bd8f81e1185d6a33a8)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:50.806594 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c33ebb3-4b2c-46e2-8fc0-d91792a46604 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:50.866919 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1020/2014] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:26:50 2022] POST /identity/v3/users => generated 289 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:26:50.872563 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3227525-6581-4a0b-8427-8fe11bcb2d96 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:50.882921 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3227525-6581-4a0b-8427-8fe11bcb2d96 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:50.887861 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1598ee31-0a85-4959-8e23-1a3cb1a73d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:50.888487 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1598ee31-0a85-4959-8e23-1a3cb1a73d4e admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:50.888768 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1598ee31-0a85-4959-8e23-1a3cb1a73d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:50.888768 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1598ee31-0a85-4959-8e23-1a3cb1a73d4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:50.888768 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1598ee31-0a85-4959-8e23-1a3cb1a73d4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:50.888768 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1598ee31-0a85-4959-8e23-1a3cb1a73d4e admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:50.902706 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1021/2015] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:26:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:26:50.906852 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1598ee31-0a85-4959-8e23-1a3cb1a73d4e admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:50.917563 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1598ee31-0a85-4959-8e23-1a3cb1a73d4e admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:50.923105 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78919fe-953f-4bef-b409-cf40e70881dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:50.924213 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78919fe-953f-4bef-b409-cf40e70881dc admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:50.924213 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78919fe-953f-4bef-b409-cf40e70881dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:50.924452 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78919fe-953f-4bef-b409-cf40e70881dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:50.925010 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78919fe-953f-4bef-b409-cf40e70881dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:50.926650 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78919fe-953f-4bef-b409-cf40e70881dc admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:50.941968 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1022/2016] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:26:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:26:50.942605 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-5c33ebb3-4b2c-46e2-8fc0-d91792a46604 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33e98e2129fe47bd8f81e1185d6a33a8', 'request_id': 'req-5c33ebb3-4b2c-46e2-8fc0-d91792a46604'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:50.943411 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78919fe-953f-4bef-b409-cf40e70881dc admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:50.949257 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-5c33ebb3-4b2c-46e2-8fc0-d91792a46604 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33e98e2129fe47bd8f81e1185d6a33a8', 'request_id': 'req-5c33ebb3-4b2c-46e2-8fc0-d91792a46604'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:50.955224 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78919fe-953f-4bef-b409-cf40e70881dc admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:50.997734 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-5c33ebb3-4b2c-46e2-8fc0-d91792a46604 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33e98e2129fe47bd8f81e1185d6a33a8', 'request_id': 'req-5c33ebb3-4b2c-46e2-8fc0-d91792a46604'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:51.012266 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-5c33ebb3-4b2c-46e2-8fc0-d91792a46604 admin admin] The token cache is being invalidate because project 33e98e2129fe47bd8f81e1185d6a33a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:26:51.012626 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-5c33ebb3-4b2c-46e2-8fc0-d91792a46604 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:51.013963 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-5c33ebb3-4b2c-46e2-8fc0-d91792a46604 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:51.016078 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 995/2017] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:26:50 2022] DELETE /identity/v3/projects/33e98e2129fe47bd8f81e1185d6a33a8 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:26:51.022003 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22c1911-1f39-46eb-90ed-50858fe5294b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:51.022689 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22c1911-1f39-46eb-90ed-50858fe5294b admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:51.022980 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22c1911-1f39-46eb-90ed-50858fe5294b admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:51.023259 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22c1911-1f39-46eb-90ed-50858fe5294b admin admin] PATH_INFO: `/v3/projects/63581b8c221a497897c537bd6e36b83e/users/e088edbc1bfe4b13ae6481f7ef592f74/roles/8feebc60363c4feb870718b1721e5324` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:51.028564 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497ae511-3421-4a26-960e-f7b300bfc4b2 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:51.028865 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497ae511-3421-4a26-960e-f7b300bfc4b2 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:51.029133 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497ae511-3421-4a26-960e-f7b300bfc4b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:51.039811 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22c1911-1f39-46eb-90ed-50858fe5294b admin admin] RBAC: Authorizing `identity:create_grant(project_id=63581b8c221a497897c537bd6e36b83e, user_id=e088edbc1bfe4b13ae6481f7ef592f74, role_id=8feebc60363c4feb870718b1721e5324)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:51.042403 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22c1911-1f39-46eb-90ed-50858fe5294b admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:51.052478 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-497ae511-3421-4a26-960e-f7b300bfc4b2 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:26:51.053743 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-497ae511-3421-4a26-960e-f7b300bfc4b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:51.061958 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 996/2018] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:26:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:26:51.070630 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5bf669-2bbc-492d-bf15-a904ad91d3c2 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:51.070919 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5bf669-2bbc-492d-bf15-a904ad91d3c2 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:51.071190 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5bf669-2bbc-492d-bf15-a904ad91d3c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:51.090968 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-3c5bf669-2bbc-492d-bf15-a904ad91d3c2 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:26:51.092201 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-3c5bf669-2bbc-492d-bf15-a904ad91d3c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:51.100297 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 997/2019] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:26:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:26:51.104510 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5bf669-2bbc-492d-bf15-a904ad91d3c2 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:51.114762 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5bf669-2bbc-492d-bf15-a904ad91d3c2 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:51.119570 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcb2a16-af39-4b87-b920-c50c0a0fbc3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:51.120314 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcb2a16-af39-4b87-b920-c50c0a0fbc3e admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:51.120628 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcb2a16-af39-4b87-b920-c50c0a0fbc3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:51.120990 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcb2a16-af39-4b87-b920-c50c0a0fbc3e admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:51.121755 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcb2a16-af39-4b87-b920-c50c0a0fbc3e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:26:51.122169 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcb2a16-af39-4b87-b920-c50c0a0fbc3e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:51.123889 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcb2a16-af39-4b87-b920-c50c0a0fbc3e admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:51.132339 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1023/2020] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:26:50 2022] PUT /identity/v3/projects/63581b8c221a497897c537bd6e36b83e/users/e088edbc1bfe4b13ae6481f7ef592f74/roles/8feebc60363c4feb870718b1721e5324 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:26:51.138370 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 998/2021] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Aug 4 21:26:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:26:51.142800 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22c1911-1f39-46eb-90ed-50858fe5294b admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:51.143340 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6feb7ae-156b-4e31-9b11-c6116af362b4 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:51.143771 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6feb7ae-156b-4e31-9b11-c6116af362b4 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:51.144128 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6feb7ae-156b-4e31-9b11-c6116af362b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:51.169308 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:26:51.178860 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-c6feb7ae-156b-4e31-9b11-c6116af362b4 None None] MFA Rules not processed for user `e088edbc1bfe4b13ae6481f7ef592f74`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:26:51.180124 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-c6feb7ae-156b-4e31-9b11-c6116af362b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:51.216341 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22c1911-1f39-46eb-90ed-50858fe5294b admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:51.223648 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615a074b-e245-45a6-837f-19e2fa50144a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:51.224419 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615a074b-e245-45a6-837f-19e2fa50144a admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:51.224769 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615a074b-e245-45a6-837f-19e2fa50144a admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:51.229399 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615a074b-e245-45a6-837f-19e2fa50144a admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:51.229399 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615a074b-e245-45a6-837f-19e2fa50144a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:51.229399 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615a074b-e245-45a6-837f-19e2fa50144a admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:51.259208 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 999/2022] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:26:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:26:51.263571 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6feb7ae-156b-4e31-9b11-c6116af362b4 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:51.264976 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-c6feb7ae-156b-4e31-9b11-c6116af362b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:51.279778 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6feb7ae-156b-4e31-9b11-c6116af362b4 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:51.285519 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d006fe9-8dca-47b2-9709-f94d5833f3a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:51.286270 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d006fe9-8dca-47b2-9709-f94d5833f3a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:51.286650 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d006fe9-8dca-47b2-9709-f94d5833f3a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:51.287022 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d006fe9-8dca-47b2-9709-f94d5833f3a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:51.287954 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d006fe9-8dca-47b2-9709-f94d5833f3a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:51.290023 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d006fe9-8dca-47b2-9709-f94d5833f3a5 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:51.325331 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1024/2023] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:26:51 2022] POST /identity/v3/projects => generated 373 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:26:51.329877 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615a074b-e245-45a6-837f-19e2fa50144a admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:51.334379 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:26:51.395275 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615a074b-e245-45a6-837f-19e2fa50144a admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:51.400545 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2065a7c1-ec5b-4f7d-943c-e7a86f0e9a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:51.401216 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2065a7c1-ec5b-4f7d-943c-e7a86f0e9a28 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:51.401499 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2065a7c1-ec5b-4f7d-943c-e7a86f0e9a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:51.401776 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2065a7c1-ec5b-4f7d-943c-e7a86f0e9a28 admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:51.402782 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2065a7c1-ec5b-4f7d-943c-e7a86f0e9a28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:51.404758 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2065a7c1-ec5b-4f7d-943c-e7a86f0e9a28 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:51.416234 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1000/2024] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:26:51 2022] POST /identity/v3/projects => generated 373 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:26:51.417322 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d006fe9-8dca-47b2-9709-f94d5833f3a5 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:51.483109 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d006fe9-8dca-47b2-9709-f94d5833f3a5 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:51.484635 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1025/2025] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:26:51 2022] POST /identity/v3/users => generated 287 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:26:51.489148 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76e7cfd-e3da-459c-a9e5-bf3e65404c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:51.489472 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2065a7c1-ec5b-4f7d-943c-e7a86f0e9a28 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:51.489843 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76e7cfd-e3da-459c-a9e5-bf3e65404c4b admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:51.490061 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76e7cfd-e3da-459c-a9e5-bf3e65404c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:51.490336 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76e7cfd-e3da-459c-a9e5-bf3e65404c4b admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:51.490999 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76e7cfd-e3da-459c-a9e5-bf3e65404c4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:51.493183 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76e7cfd-e3da-459c-a9e5-bf3e65404c4b admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:51.499365 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2065a7c1-ec5b-4f7d-943c-e7a86f0e9a28 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:51.504418 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c887e2f-ebb7-4510-a41c-955637a96319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:51.505089 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c887e2f-ebb7-4510-a41c-955637a96319 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:51.505621 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c887e2f-ebb7-4510-a41c-955637a96319 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:51.506121 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c887e2f-ebb7-4510-a41c-955637a96319 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:51.506996 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c887e2f-ebb7-4510-a41c-955637a96319 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:51.508729 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c887e2f-ebb7-4510-a41c-955637a96319 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:51.522204 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1026/2026] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:26:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:26:51.526373 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c887e2f-ebb7-4510-a41c-955637a96319 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:51.536544 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c887e2f-ebb7-4510-a41c-955637a96319 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:51.541608 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7a4270-77b7-4ac4-bda7-018785b974ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:51.542156 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7a4270-77b7-4ac4-bda7-018785b974ac admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:51.542441 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7a4270-77b7-4ac4-bda7-018785b974ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:51.542717 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7a4270-77b7-4ac4-bda7-018785b974ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:51.543443 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7a4270-77b7-4ac4-bda7-018785b974ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:51.545100 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7a4270-77b7-4ac4-bda7-018785b974ac admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:51.555872 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1027/2027] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:26:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:26:51.560103 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7a4270-77b7-4ac4-bda7-018785b974ac admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:51.572069 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7a4270-77b7-4ac4-bda7-018785b974ac admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:51.575024 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44ebbd8-d770-40a5-8630-7d866c310dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:51.575704 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44ebbd8-d770-40a5-8630-7d866c310dd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:51.575994 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44ebbd8-d770-40a5-8630-7d866c310dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:51.576268 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44ebbd8-d770-40a5-8630-7d866c310dd5 admin admin] PATH_INFO: `/v3/projects/854bedcf8b864a369cae363a24282397/users/13846f6cdd664c7bbc75c2e5e7dcafd3/roles/8feebc60363c4feb870718b1721e5324` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:51.590777 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44ebbd8-d770-40a5-8630-7d866c310dd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=854bedcf8b864a369cae363a24282397, user_id=13846f6cdd664c7bbc75c2e5e7dcafd3, role_id=8feebc60363c4feb870718b1721e5324)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:51.593476 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44ebbd8-d770-40a5-8630-7d866c310dd5 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:51.598167 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1001/2028] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:26:51 2022] POST /identity/v3/users => generated 287 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:26:51.601945 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76e7cfd-e3da-459c-a9e5-bf3e65404c4b admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:51.612467 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76e7cfd-e3da-459c-a9e5-bf3e65404c4b admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:51.617133 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdd4d8e-8530-41c9-903b-83ba8046bdb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:51.617732 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdd4d8e-8530-41c9-903b-83ba8046bdb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:51.618024 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdd4d8e-8530-41c9-903b-83ba8046bdb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:51.618298 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdd4d8e-8530-41c9-903b-83ba8046bdb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:51.618996 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdd4d8e-8530-41c9-903b-83ba8046bdb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:51.629797 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdd4d8e-8530-41c9-903b-83ba8046bdb4 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:51.634060 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1002/2029] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:26:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:26:51.635209 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdd4d8e-8530-41c9-903b-83ba8046bdb4 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:51.645227 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdd4d8e-8530-41c9-903b-83ba8046bdb4 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:51.650384 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc3aee3-31ee-474f-8f1e-89c9733dc1bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:51.651055 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc3aee3-31ee-474f-8f1e-89c9733dc1bb admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:51.651363 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc3aee3-31ee-474f-8f1e-89c9733dc1bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:51.651649 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc3aee3-31ee-474f-8f1e-89c9733dc1bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:51.652347 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc3aee3-31ee-474f-8f1e-89c9733dc1bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:51.653991 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc3aee3-31ee-474f-8f1e-89c9733dc1bb admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:51.665212 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1003/2030] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:26:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:26:51.669339 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc3aee3-31ee-474f-8f1e-89c9733dc1bb admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:51.679806 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc3aee3-31ee-474f-8f1e-89c9733dc1bb admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:51.685282 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796d73e0-5122-4215-81bd-52598821d6e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:51.686164 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796d73e0-5122-4215-81bd-52598821d6e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:51.686795 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796d73e0-5122-4215-81bd-52598821d6e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:51.686795 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796d73e0-5122-4215-81bd-52598821d6e3 admin admin] PATH_INFO: `/v3/projects/21adbe8d4e874587a366162cab416b82/users/a9a21f29f43947b194c60d8640057ed0/roles/8feebc60363c4feb870718b1721e5324` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:51.702224 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796d73e0-5122-4215-81bd-52598821d6e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21adbe8d4e874587a366162cab416b82, user_id=a9a21f29f43947b194c60d8640057ed0, role_id=8feebc60363c4feb870718b1721e5324)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:51.705760 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796d73e0-5122-4215-81bd-52598821d6e3 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:51.712597 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1028/2031] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:26:51 2022] PUT /identity/v3/projects/854bedcf8b864a369cae363a24282397/users/13846f6cdd664c7bbc75c2e5e7dcafd3/roles/8feebc60363c4feb870718b1721e5324 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:26:51.729950 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed406e19-ec18-46e2-88c0-04f5dccda9e6 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:51.730239 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed406e19-ec18-46e2-88c0-04f5dccda9e6 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:51.730513 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed406e19-ec18-46e2-88c0-04f5dccda9e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:51.758144 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-ed406e19-ec18-46e2-88c0-04f5dccda9e6 None None] MFA Rules not processed for user `13846f6cdd664c7bbc75c2e5e7dcafd3`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:26:51.759404 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-ed406e19-ec18-46e2-88c0-04f5dccda9e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:51.775359 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1004/2032] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:26:51 2022] PUT /identity/v3/projects/21adbe8d4e874587a366162cab416b82/users/a9a21f29f43947b194c60d8640057ed0/roles/8feebc60363c4feb870718b1721e5324 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:26:51.786700 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b989a2-d26e-4161-9001-b57a3069b13c None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:51.787062 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b989a2-d26e-4161-9001-b57a3069b13c None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:51.787393 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b989a2-d26e-4161-9001-b57a3069b13c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:51.822162 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-f1b989a2-d26e-4161-9001-b57a3069b13c None None] MFA Rules not processed for user `a9a21f29f43947b194c60d8640057ed0`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:26:51.822162 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-f1b989a2-d26e-4161-9001-b57a3069b13c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:51.826839 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:26:51.831059 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1029/2033] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:26:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:26:51.839221 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed406e19-ec18-46e2-88c0-04f5dccda9e6 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:51.885496 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:26:51.886061 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1005/2034] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:26:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:26:51.890608 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b989a2-d26e-4161-9001-b57a3069b13c None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:51.933172 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed406e19-ec18-46e2-88c0-04f5dccda9e6 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:51.968856 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b989a2-d26e-4161-9001-b57a3069b13c None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:51.974630 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c806c0-e9f5-4524-abac-d5604b43dbaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:51.977242 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c806c0-e9f5-4524-abac-d5604b43dbaa admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:51.977242 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c806c0-e9f5-4524-abac-d5604b43dbaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:51.977242 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c806c0-e9f5-4524-abac-d5604b43dbaa admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:51.977242 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c806c0-e9f5-4524-abac-d5604b43dbaa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:51.978750 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c806c0-e9f5-4524-abac-d5604b43dbaa admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:52.006939 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e05db36-f3c5-4544-9936-1685039e98c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:52.007621 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e05db36-f3c5-4544-9936-1685039e98c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:52.010196 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e05db36-f3c5-4544-9936-1685039e98c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:52.010196 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e05db36-f3c5-4544-9936-1685039e98c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:52.010196 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e05db36-f3c5-4544-9936-1685039e98c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:52.011700 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e05db36-f3c5-4544-9936-1685039e98c3 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:52.068168 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1006/2035] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:26:51 2022] POST /identity/v3/projects => generated 375 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:26:52.072325 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c806c0-e9f5-4524-abac-d5604b43dbaa admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:52.130922 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1030/2036] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:26:51 2022] POST /identity/v3/projects => generated 369 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:26:52.135021 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e05db36-f3c5-4544-9936-1685039e98c3 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:52.138125 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c806c0-e9f5-4524-abac-d5604b43dbaa admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:52.182307 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:26:52.198463 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44cac40-5bf9-4550-8e9c-4cb06fba129b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:52.199100 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44cac40-5bf9-4550-8e9c-4cb06fba129b admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:52.199915 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44cac40-5bf9-4550-8e9c-4cb06fba129b admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:52.200205 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44cac40-5bf9-4550-8e9c-4cb06fba129b admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:52.200879 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44cac40-5bf9-4550-8e9c-4cb06fba129b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:52.202871 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44cac40-5bf9-4550-8e9c-4cb06fba129b admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:52.203207 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e05db36-f3c5-4544-9936-1685039e98c3 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:52.208893 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31c0844-30bc-4b16-be5b-377b99244ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:52.213517 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31c0844-30bc-4b16-be5b-377b99244ffa admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:52.213517 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31c0844-30bc-4b16-be5b-377b99244ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:52.213517 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31c0844-30bc-4b16-be5b-377b99244ffa admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:52.213517 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31c0844-30bc-4b16-be5b-377b99244ffa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:52.213517 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31c0844-30bc-4b16-be5b-377b99244ffa admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:52.289558 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1031/2037] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:26:52 2022] POST /identity/v3/users => generated 291 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:26:52.289781 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1007/2038] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:26:52 2022] POST /identity/v3/users => generated 294 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:26:52.293737 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31c0844-30bc-4b16-be5b-377b99244ffa admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:52.294585 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44cac40-5bf9-4550-8e9c-4cb06fba129b admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:52.303511 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31c0844-30bc-4b16-be5b-377b99244ffa admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:52.305889 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44cac40-5bf9-4550-8e9c-4cb06fba129b admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:52.310319 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e7fb7d-aaf4-4c08-bab1-883f771d859a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:52.310939 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e7fb7d-aaf4-4c08-bab1-883f771d859a admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:52.311220 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e7fb7d-aaf4-4c08-bab1-883f771d859a admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:52.311344 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddece662-bbcb-4af8-b2b4-b41c9aeec938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:52.311524 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e7fb7d-aaf4-4c08-bab1-883f771d859a admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:52.311894 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddece662-bbcb-4af8-b2b4-b41c9aeec938 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:52.312167 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddece662-bbcb-4af8-b2b4-b41c9aeec938 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:52.312290 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e7fb7d-aaf4-4c08-bab1-883f771d859a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:52.312446 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddece662-bbcb-4af8-b2b4-b41c9aeec938 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:52.313992 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddece662-bbcb-4af8-b2b4-b41c9aeec938 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:52.314558 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e7fb7d-aaf4-4c08-bab1-883f771d859a admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:52.316136 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddece662-bbcb-4af8-b2b4-b41c9aeec938 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:52.325294 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1032/2039] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:26:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:26:52.330321 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e7fb7d-aaf4-4c08-bab1-883f771d859a admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:52.331012 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1008/2040] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:26:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:26:52.331904 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddece662-bbcb-4af8-b2b4-b41c9aeec938 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:52.340306 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e7fb7d-aaf4-4c08-bab1-883f771d859a admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:52.342819 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddece662-bbcb-4af8-b2b4-b41c9aeec938 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:52.344913 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f9ab62-c245-4f91-9998-63851233cbf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:52.345572 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f9ab62-c245-4f91-9998-63851233cbf9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:52.345854 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f9ab62-c245-4f91-9998-63851233cbf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:52.346128 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f9ab62-c245-4f91-9998-63851233cbf9 admin admin] PATH_INFO: `/v3/projects/059492435a0045d19f7a838cb5d01e48/users/0c78d17525264066af7f9dfb4c3ea51a/roles/a432dd4735944ecfab2fa5ce1705d356` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:52.346682 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b716679-f59e-4aa8-ad10-fa06196fecc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:52.347360 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b716679-f59e-4aa8-ad10-fa06196fecc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:52.347648 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b716679-f59e-4aa8-ad10-fa06196fecc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:52.347922 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b716679-f59e-4aa8-ad10-fa06196fecc0 admin admin] PATH_INFO: `/v3/projects/a14dbec5278e433dbcbea9d68deb7a92/users/1dad2b75f33f4424a04b64563adffbb7/roles/a432dd4735944ecfab2fa5ce1705d356` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:52.363044 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b716679-f59e-4aa8-ad10-fa06196fecc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a14dbec5278e433dbcbea9d68deb7a92, user_id=1dad2b75f33f4424a04b64563adffbb7, role_id=a432dd4735944ecfab2fa5ce1705d356)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:52.363292 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f9ab62-c245-4f91-9998-63851233cbf9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=059492435a0045d19f7a838cb5d01e48, user_id=0c78d17525264066af7f9dfb4c3ea51a, role_id=a432dd4735944ecfab2fa5ce1705d356)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:52.365770 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b716679-f59e-4aa8-ad10-fa06196fecc0 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:52.366088 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f9ab62-c245-4f91-9998-63851233cbf9 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:52.444490 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1009/2041] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:26:52 2022] PUT /identity/v3/projects/a14dbec5278e433dbcbea9d68deb7a92/users/1dad2b75f33f4424a04b64563adffbb7/roles/a432dd4735944ecfab2fa5ce1705d356 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:26:52.448610 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b716679-f59e-4aa8-ad10-fa06196fecc0 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:52.467191 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1033/2042] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:26:52 2022] PUT /identity/v3/projects/059492435a0045d19f7a838cb5d01e48/users/0c78d17525264066af7f9dfb4c3ea51a/roles/a432dd4735944ecfab2fa5ce1705d356 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:26:52.472805 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f9ab62-c245-4f91-9998-63851233cbf9 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:52.501505 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:26:52.516100 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b716679-f59e-4aa8-ad10-fa06196fecc0 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:52.539507 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f9ab62-c245-4f91-9998-63851233cbf9 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:52.544471 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab09bb4b-e05b-4ff3-bbaf-c493c3e9ad42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:52.545093 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab09bb4b-e05b-4ff3-bbaf-c493c3e9ad42 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:52.545378 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab09bb4b-e05b-4ff3-bbaf-c493c3e9ad42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:52.546374 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab09bb4b-e05b-4ff3-bbaf-c493c3e9ad42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:52.546374 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab09bb4b-e05b-4ff3-bbaf-c493c3e9ad42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:52.548514 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab09bb4b-e05b-4ff3-bbaf-c493c3e9ad42 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:52.562871 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1034/2043] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:26:52 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:26:52.564771 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab09bb4b-e05b-4ff3-bbaf-c493c3e9ad42 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:52.574336 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab09bb4b-e05b-4ff3-bbaf-c493c3e9ad42 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:52.579010 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d43ef02-775a-474c-ab37-ffad74c2651b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:52.579428 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57a4dc8-1d90-47d6-804f-7a7b3fe003d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:52.579638 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d43ef02-775a-474c-ab37-ffad74c2651b admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:52.579921 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d43ef02-775a-474c-ab37-ffad74c2651b admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:52.580123 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57a4dc8-1d90-47d6-804f-7a7b3fe003d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:52.580243 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d43ef02-775a-474c-ab37-ffad74c2651b admin admin] PATH_INFO: `/v3/projects/059492435a0045d19f7a838cb5d01e48/users/0c78d17525264066af7f9dfb4c3ea51a/roles/a432dd4735944ecfab2fa5ce1705d356` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:52.580436 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57a4dc8-1d90-47d6-804f-7a7b3fe003d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:52.580680 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57a4dc8-1d90-47d6-804f-7a7b3fe003d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:52.581451 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57a4dc8-1d90-47d6-804f-7a7b3fe003d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:52.582141 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d43ef02-775a-474c-ab37-ffad74c2651b admin admin] RBAC: Authorizing `identity:create_grant(project_id=059492435a0045d19f7a838cb5d01e48, user_id=0c78d17525264066af7f9dfb4c3ea51a, role_id=a432dd4735944ecfab2fa5ce1705d356)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:52.583082 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57a4dc8-1d90-47d6-804f-7a7b3fe003d5 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:52.584997 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d43ef02-775a-474c-ab37-ffad74c2651b admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:52.593437 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1010/2044] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:26:52 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:26:52.595495 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1035/2045] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:26:52 2022] PUT /identity/v3/projects/059492435a0045d19f7a838cb5d01e48/users/0c78d17525264066af7f9dfb4c3ea51a/roles/a432dd4735944ecfab2fa5ce1705d356 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:26:52.598049 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57a4dc8-1d90-47d6-804f-7a7b3fe003d5 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:52.607114 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3343fddf-be3c-4bca-b067-46a673c4c353 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:52.607739 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3343fddf-be3c-4bca-b067-46a673c4c353 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:52.608036 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3343fddf-be3c-4bca-b067-46a673c4c353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:52.609086 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57a4dc8-1d90-47d6-804f-7a7b3fe003d5 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:52.638797 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-3343fddf-be3c-4bca-b067-46a673c4c353 None None] MFA Rules not processed for user `0c78d17525264066af7f9dfb4c3ea51a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:26:52.640064 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-3343fddf-be3c-4bca-b067-46a673c4c353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:52.653347 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:26:52.669830 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658017ae-1bb1-4b75-a682-273b556003b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:52.670517 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658017ae-1bb1-4b75-a682-273b556003b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:52.670799 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658017ae-1bb1-4b75-a682-273b556003b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:52.671072 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658017ae-1bb1-4b75-a682-273b556003b0 admin admin] PATH_INFO: `/v3/projects/a14dbec5278e433dbcbea9d68deb7a92/users/1dad2b75f33f4424a04b64563adffbb7/roles/a432dd4735944ecfab2fa5ce1705d356` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:52.673684 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658017ae-1bb1-4b75-a682-273b556003b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a14dbec5278e433dbcbea9d68deb7a92, user_id=1dad2b75f33f4424a04b64563adffbb7, role_id=a432dd4735944ecfab2fa5ce1705d356)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:52.676436 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658017ae-1bb1-4b75-a682-273b556003b0 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:52.686750 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1011/2046] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:26:52 2022] PUT /identity/v3/projects/a14dbec5278e433dbcbea9d68deb7a92/users/1dad2b75f33f4424a04b64563adffbb7/roles/a432dd4735944ecfab2fa5ce1705d356 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:26:52.698757 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f579444-a8c7-468c-a784-54f764b20b41 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:52.699046 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f579444-a8c7-468c-a784-54f764b20b41 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:52.699339 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f579444-a8c7-468c-a784-54f764b20b41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:52.714343 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1036/2047] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:26:52 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:26:52.726952 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3343fddf-be3c-4bca-b067-46a673c4c353 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:52.728857 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-3343fddf-be3c-4bca-b067-46a673c4c353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:52.730076 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-1f579444-a8c7-468c-a784-54f764b20b41 None None] MFA Rules not processed for user `1dad2b75f33f4424a04b64563adffbb7`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:26:52.731470 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-1f579444-a8c7-468c-a784-54f764b20b41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:52.786774 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3343fddf-be3c-4bca-b067-46a673c4c353 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:52.803170 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1012/2048] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:26:52 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:26:52.815617 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f579444-a8c7-468c-a784-54f764b20b41 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:52.817909 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-1f579444-a8c7-468c-a784-54f764b20b41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:52.839890 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:26:52.845177 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00625274-0338-4337-ac51-015cd5782549 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e31f2728056b460fa474478eb85ca1d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:52.845805 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00625274-0338-4337-ac51-015cd5782549 service cinder] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:52.846092 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00625274-0338-4337-ac51-015cd5782549 service cinder] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:52.846369 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00625274-0338-4337-ac51-015cd5782549 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:52.881079 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f579444-a8c7-468c-a784-54f764b20b41 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:52.884946 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8581e5a0-ab0d-466a-8b32-9bb61162d70f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e31f2728056b460fa474478eb85ca1d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:52.885588 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8581e5a0-ab0d-466a-8b32-9bb61162d70f service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:52.885894 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8581e5a0-ab0d-466a-8b32-9bb61162d70f service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:52.886174 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8581e5a0-ab0d-466a-8b32-9bb61162d70f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:52.912075 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00625274-0338-4337-ac51-015cd5782549 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:52.913528 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00625274-0338-4337-ac51-015cd5782549 service cinder] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:52.924201 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1037/2049] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:26:52 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:26:52.944532 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8581e5a0-ab0d-466a-8b32-9bb61162d70f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:52.946061 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8581e5a0-ab0d-466a-8b32-9bb61162d70f service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:52.955533 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1013/2050] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:26:52 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:26:53.201100 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00625274-0338-4337-ac51-015cd5782549 service cinder] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:53.209631 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00625274-0338-4337-ac51-015cd5782549 service cinder] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:53.213665 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91409423-8a14-455a-b619-e9ceff29d5c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e31f2728056b460fa474478eb85ca1d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:53.214112 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91409423-8a14-455a-b619-e9ceff29d5c3 service cinder] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:53.214396 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91409423-8a14-455a-b619-e9ceff29d5c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:53.214727 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91409423-8a14-455a-b619-e9ceff29d5c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:53.278441 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91409423-8a14-455a-b619-e9ceff29d5c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:53.279923 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91409423-8a14-455a-b619-e9ceff29d5c3 service cinder] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:53.290124 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1038/2051] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:26:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:26:54.908964 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8581e5a0-ab0d-466a-8b32-9bb61162d70f service cinder] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:54.923052 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8581e5a0-ab0d-466a-8b32-9bb61162d70f service cinder] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:54.926742 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d9812f-2d51-4d58-aeb4-ff122413d669 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e31f2728056b460fa474478eb85ca1d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:54.927400 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d9812f-2d51-4d58-aeb4-ff122413d669 service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:54.927711 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d9812f-2d51-4d58-aeb4-ff122413d669 service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:54.927966 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d9812f-2d51-4d58-aeb4-ff122413d669 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:54.941491 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:26:54.996262 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d9812f-2d51-4d58-aeb4-ff122413d669 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:54.997909 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d9812f-2d51-4d58-aeb4-ff122413d669 service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:55.007490 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1014/2052] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:26:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:26:55.786894 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91409423-8a14-455a-b619-e9ceff29d5c3 service cinder] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:55.788755 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-91409423-8a14-455a-b619-e9ceff29d5c3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:55.857262 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91409423-8a14-455a-b619-e9ceff29d5c3 service cinder] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:55.862501 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc319558-ccdb-4bed-9d27-54c6e4fa98e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:55.863173 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc319558-ccdb-4bed-9d27-54c6e4fa98e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:55.863486 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc319558-ccdb-4bed-9d27-54c6e4fa98e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:55.863761 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc319558-ccdb-4bed-9d27-54c6e4fa98e1 admin admin] PATH_INFO: `/v3/users/948c7bacbadc4676ba75b1d29e6c0bd1` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:55.864928 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc319558-ccdb-4bed-9d27-54c6e4fa98e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=948c7bacbadc4676ba75b1d29e6c0bd1)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:55.866906 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc319558-ccdb-4bed-9d27-54c6e4fa98e1 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:56.072604 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:26:56.082185 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-cc319558-ccdb-4bed-9d27-54c6e4fa98e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '948c7bacbadc4676ba75b1d29e6c0bd1'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:56.094432 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-cc319558-ccdb-4bed-9d27-54c6e4fa98e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '948c7bacbadc4676ba75b1d29e6c0bd1'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:56.095228 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-cc319558-ccdb-4bed-9d27-54c6e4fa98e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '948c7bacbadc4676ba75b1d29e6c0bd1'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:56.095874 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-cc319558-ccdb-4bed-9d27-54c6e4fa98e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '948c7bacbadc4676ba75b1d29e6c0bd1'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:56.191758 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-cc319558-ccdb-4bed-9d27-54c6e4fa98e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '948c7bacbadc4676ba75b1d29e6c0bd1'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:56.205906 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1039/2053] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:26:55 2022] DELETE /identity/v3/users/948c7bacbadc4676ba75b1d29e6c0bd1 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:26:56.210451 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c2c674-6811-446d-a3e5-fc1e3220b022 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:56.210969 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c2c674-6811-446d-a3e5-fc1e3220b022 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:56.211091 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c2c674-6811-446d-a3e5-fc1e3220b022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:56.236361 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-b7c2c674-6811-446d-a3e5-fc1e3220b022 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:26:56.237731 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-b7c2c674-6811-446d-a3e5-fc1e3220b022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:56.309166 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1015/2054] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:26:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:26:56.321495 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc319558-ccdb-4bed-9d27-54c6e4fa98e1 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:56.323001 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-cc319558-ccdb-4bed-9d27-54c6e4fa98e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:56.385024 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c2c674-6811-446d-a3e5-fc1e3220b022 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:56.386504 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-b7c2c674-6811-446d-a3e5-fc1e3220b022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:56.391591 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc319558-ccdb-4bed-9d27-54c6e4fa98e1 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:56.395563 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a901ca1-5e47-432a-8c72-a2fdd795264d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:56.396345 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a901ca1-5e47-432a-8c72-a2fdd795264d service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:56.396660 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a901ca1-5e47-432a-8c72-a2fdd795264d service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:56.396938 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a901ca1-5e47-432a-8c72-a2fdd795264d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:56.408308 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a901ca1-5e47-432a-8c72-a2fdd795264d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:56.409744 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a901ca1-5e47-432a-8c72-a2fdd795264d service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:56.420215 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1040/2055] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:26:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:26:56.451245 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c2c674-6811-446d-a3e5-fc1e3220b022 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:56.455062 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559b4d52-e1e1-42fe-91ed-601405ba8449 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e31f2728056b460fa474478eb85ca1d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:56.455733 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b4d52-e1e1-42fe-91ed-601405ba8449 service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:56.456009 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b4d52-e1e1-42fe-91ed-601405ba8449 service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:56.456282 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b4d52-e1e1-42fe-91ed-601405ba8449 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:56.458281 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-559b4d52-e1e1-42fe-91ed-601405ba8449 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:56.524292 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559b4d52-e1e1-42fe-91ed-601405ba8449 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:56.525879 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559b4d52-e1e1-42fe-91ed-601405ba8449 service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:56.535773 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1016/2056] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:26:56 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:26:56.868656 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a901ca1-5e47-432a-8c72-a2fdd795264d service neutron] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:56.870253 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-9a901ca1-5e47-432a-8c72-a2fdd795264d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:56.894271 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a901ca1-5e47-432a-8c72-a2fdd795264d service neutron] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:56.901384 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3416e7cf-35d0-42da-9a54-27417505c73e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:56.901384 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3416e7cf-35d0-42da-9a54-27417505c73e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:56.901384 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3416e7cf-35d0-42da-9a54-27417505c73e admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:56.901384 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3416e7cf-35d0-42da-9a54-27417505c73e admin admin] PATH_INFO: `/v3/projects/eab4594071a64f71aa3f31735de103d3` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:56.902374 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3416e7cf-35d0-42da-9a54-27417505c73e admin admin] RBAC: Authorizing `identity:delete_project(project_id=eab4594071a64f71aa3f31735de103d3)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:56.903565 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3416e7cf-35d0-42da-9a54-27417505c73e admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:57.069717 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-3416e7cf-35d0-42da-9a54-27417505c73e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eab4594071a64f71aa3f31735de103d3', 'request_id': 'req-3416e7cf-35d0-42da-9a54-27417505c73e'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:57.075574 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-3416e7cf-35d0-42da-9a54-27417505c73e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eab4594071a64f71aa3f31735de103d3', 'request_id': 'req-3416e7cf-35d0-42da-9a54-27417505c73e'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:57.169149 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-3416e7cf-35d0-42da-9a54-27417505c73e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eab4594071a64f71aa3f31735de103d3', 'request_id': 'req-3416e7cf-35d0-42da-9a54-27417505c73e'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:57.183173 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-3416e7cf-35d0-42da-9a54-27417505c73e admin admin] The token cache is being invalidate because project eab4594071a64f71aa3f31735de103d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:26:57.183500 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-3416e7cf-35d0-42da-9a54-27417505c73e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:57.184231 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-3416e7cf-35d0-42da-9a54-27417505c73e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:26:57.185982 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1041/2057] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:26:56 2022] DELETE /identity/v3/projects/eab4594071a64f71aa3f31735de103d3 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:26:57.191196 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee66052-5147-45f7-95ec-dc8d93365b87 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:57.191563 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee66052-5147-45f7-95ec-dc8d93365b87 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:57.191845 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee66052-5147-45f7-95ec-dc8d93365b87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:57.201828 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:26:57.214285 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-2ee66052-5147-45f7-95ec-dc8d93365b87 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:26:57.215539 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-2ee66052-5147-45f7-95ec-dc8d93365b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:57.280376 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1017/2058] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:26:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:26:57.284148 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb302ff-ce12-4764-a390-5227abafddb1 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:57.284439 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb302ff-ce12-4764-a390-5227abafddb1 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:57.285305 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb302ff-ce12-4764-a390-5227abafddb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:57.306471 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-0cb302ff-ce12-4764-a390-5227abafddb1 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:26:57.307721 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-0cb302ff-ce12-4764-a390-5227abafddb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:57.316243 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1042/2059] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:26:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:26:57.320597 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee66052-5147-45f7-95ec-dc8d93365b87 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:57.331069 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee66052-5147-45f7-95ec-dc8d93365b87 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:57.335726 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e79ba7-cb53-4cb6-a34c-cc9fc8721af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:57.336341 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e79ba7-cb53-4cb6-a34c-cc9fc8721af6 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:57.336666 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e79ba7-cb53-4cb6-a34c-cc9fc8721af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:57.336943 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e79ba7-cb53-4cb6-a34c-cc9fc8721af6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:57.337639 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e79ba7-cb53-4cb6-a34c-cc9fc8721af6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:26:57.337944 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e79ba7-cb53-4cb6-a34c-cc9fc8721af6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:57.339655 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e79ba7-cb53-4cb6-a34c-cc9fc8721af6 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:57.352273 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1018/2060] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Aug 4 21:26:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:26:57.357359 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb302ff-ce12-4764-a390-5227abafddb1 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:57.369872 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb302ff-ce12-4764-a390-5227abafddb1 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:57.375548 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4fa197-60fc-4e90-bee0-7492dad14179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:57.375927 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4fa197-60fc-4e90-bee0-7492dad14179 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:57.376637 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4fa197-60fc-4e90-bee0-7492dad14179 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:57.376907 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4fa197-60fc-4e90-bee0-7492dad14179 admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:57.377580 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4fa197-60fc-4e90-bee0-7492dad14179 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:57.379584 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4fa197-60fc-4e90-bee0-7492dad14179 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:57.473128 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1043/2061] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:26:57 2022] POST /identity/v3/projects => generated 365 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:26:57.478378 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e79ba7-cb53-4cb6-a34c-cc9fc8721af6 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:57.488747 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e79ba7-cb53-4cb6-a34c-cc9fc8721af6 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:57.547553 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d900896a-a165-4e6b-8622-0765abc745c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:57.548180 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d900896a-a165-4e6b-8622-0765abc745c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:57.548466 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d900896a-a165-4e6b-8622-0765abc745c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:57.548749 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d900896a-a165-4e6b-8622-0765abc745c3 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:57.549415 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d900896a-a165-4e6b-8622-0765abc745c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:57.551444 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d900896a-a165-4e6b-8622-0765abc745c3 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:57.677881 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1019/2062] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:26:57 2022] POST /identity/v3/users => generated 283 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:26:57.682043 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4fa197-60fc-4e90-bee0-7492dad14179 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:57.693095 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4fa197-60fc-4e90-bee0-7492dad14179 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:57.697636 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea9803c-0b57-4c64-94da-328a78337bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:57.703341 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea9803c-0b57-4c64-94da-328a78337bb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:57.703341 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea9803c-0b57-4c64-94da-328a78337bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:57.703341 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea9803c-0b57-4c64-94da-328a78337bb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:57.703341 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea9803c-0b57-4c64-94da-328a78337bb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:57.703341 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea9803c-0b57-4c64-94da-328a78337bb4 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:57.707583 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:26:57.712515 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1044/2063] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:26:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:26:57.716696 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d900896a-a165-4e6b-8622-0765abc745c3 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:57.726707 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d900896a-a165-4e6b-8622-0765abc745c3 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:57.731301 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f960ec-66bf-455d-a2d1-44fe86546b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:57.731924 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f960ec-66bf-455d-a2d1-44fe86546b80 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:57.732206 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f960ec-66bf-455d-a2d1-44fe86546b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:57.732470 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f960ec-66bf-455d-a2d1-44fe86546b80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:57.733193 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f960ec-66bf-455d-a2d1-44fe86546b80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:57.734908 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f960ec-66bf-455d-a2d1-44fe86546b80 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:57.746186 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1020/2064] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:26:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:26:57.750367 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea9803c-0b57-4c64-94da-328a78337bb4 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:57.761416 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea9803c-0b57-4c64-94da-328a78337bb4 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:57.766295 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e54c7c-c0b8-4518-b76d-9aa3008be3f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:57.767585 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e54c7c-c0b8-4518-b76d-9aa3008be3f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:57.767919 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e54c7c-c0b8-4518-b76d-9aa3008be3f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:57.768223 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e54c7c-c0b8-4518-b76d-9aa3008be3f1 admin admin] PATH_INFO: `/v3/projects/159668eb78014f06ae1940436ca305bf/users/d895fcc24e4543aca8b23ae761630417/roles/8feebc60363c4feb870718b1721e5324` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:57.786294 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e54c7c-c0b8-4518-b76d-9aa3008be3f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=159668eb78014f06ae1940436ca305bf, user_id=d895fcc24e4543aca8b23ae761630417, role_id=8feebc60363c4feb870718b1721e5324)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:57.790970 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e54c7c-c0b8-4518-b76d-9aa3008be3f1 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:57.917715 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1045/2065] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:26:57 2022] PUT /identity/v3/projects/159668eb78014f06ae1940436ca305bf/users/d895fcc24e4543aca8b23ae761630417/roles/8feebc60363c4feb870718b1721e5324 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:26:57.929769 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d188f0-b138-4f22-890c-1c2729fc9476 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:57.930464 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d188f0-b138-4f22-890c-1c2729fc9476 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:57.930464 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d188f0-b138-4f22-890c-1c2729fc9476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:57.960636 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-80d188f0-b138-4f22-890c-1c2729fc9476 None None] MFA Rules not processed for user `d895fcc24e4543aca8b23ae761630417`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:26:57.961855 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-80d188f0-b138-4f22-890c-1c2729fc9476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:58.025033 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:26:58.028772 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1021/2066] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:26:57 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:26:58.040013 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e54c7c-c0b8-4518-b76d-9aa3008be3f1 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:26:58.041985 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-78e54c7c-c0b8-4518-b76d-9aa3008be3f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:26:58.104058 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e54c7c-c0b8-4518-b76d-9aa3008be3f1 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:26:58.108039 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172939c5-187a-490e-9d07-450fcc9cb4fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e31f2728056b460fa474478eb85ca1d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:26:58.108663 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172939c5-187a-490e-9d07-450fcc9cb4fd service cinder] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:26:58.108948 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172939c5-187a-490e-9d07-450fcc9cb4fd service cinder] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:26:58.109227 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172939c5-187a-490e-9d07-450fcc9cb4fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:26:58.118003 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172939c5-187a-490e-9d07-450fcc9cb4fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:26:58.119470 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172939c5-187a-490e-9d07-450fcc9cb4fd service cinder] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:26:58.128922 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1046/2067] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:26:58 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:27:03.901855 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d188f0-b138-4f22-890c-1c2729fc9476 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:27:03.903354 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-80d188f0-b138-4f22-890c-1c2729fc9476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:27:03.976853 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d188f0-b138-4f22-890c-1c2729fc9476 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:27:03.982255 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ba8431-0aad-49da-bedc-d8fac055d31f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:27:03.983002 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ba8431-0aad-49da-bedc-d8fac055d31f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:03.983304 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ba8431-0aad-49da-bedc-d8fac055d31f admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:03.983677 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ba8431-0aad-49da-bedc-d8fac055d31f admin admin] PATH_INFO: `/v3/users/e088edbc1bfe4b13ae6481f7ef592f74` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:03.984854 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ba8431-0aad-49da-bedc-d8fac055d31f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e088edbc1bfe4b13ae6481f7ef592f74)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:27:03.987596 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ba8431-0aad-49da-bedc-d8fac055d31f admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:27:04.183442 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-58ba8431-0aad-49da-bedc-d8fac055d31f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e088edbc1bfe4b13ae6481f7ef592f74'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:04.196457 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-58ba8431-0aad-49da-bedc-d8fac055d31f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e088edbc1bfe4b13ae6481f7ef592f74'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:04.197414 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-58ba8431-0aad-49da-bedc-d8fac055d31f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e088edbc1bfe4b13ae6481f7ef592f74'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:04.204384 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-58ba8431-0aad-49da-bedc-d8fac055d31f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e088edbc1bfe4b13ae6481f7ef592f74'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:04.205095 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-58ba8431-0aad-49da-bedc-d8fac055d31f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e088edbc1bfe4b13ae6481f7ef592f74'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:04.277954 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1022/2068] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:27:03 2022] DELETE /identity/v3/users/e088edbc1bfe4b13ae6481f7ef592f74 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:27:04.282438 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4efc03-3df4-4f62-a61b-524faf99d4b9 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:04.283562 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4efc03-3df4-4f62-a61b-524faf99d4b9 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:04.283846 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4efc03-3df4-4f62-a61b-524faf99d4b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:04.310892 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-8d4efc03-3df4-4f62-a61b-524faf99d4b9 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:27:04.311184 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-8d4efc03-3df4-4f62-a61b-524faf99d4b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:27:04.395335 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1047/2069] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:27:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:27:04.406027 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ba8431-0aad-49da-bedc-d8fac055d31f admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:27:04.407607 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-58ba8431-0aad-49da-bedc-d8fac055d31f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:27:04.474181 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ba8431-0aad-49da-bedc-d8fac055d31f admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:27:04.480270 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fbbf5b-a1b4-420d-acb6-51919f87a2f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:27:04.480270 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fbbf5b-a1b4-420d-acb6-51919f87a2f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:04.480270 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fbbf5b-a1b4-420d-acb6-51919f87a2f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:04.480270 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fbbf5b-a1b4-420d-acb6-51919f87a2f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:04.490037 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fbbf5b-a1b4-420d-acb6-51919f87a2f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:27:04.491513 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fbbf5b-a1b4-420d-acb6-51919f87a2f7 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:27:04.501187 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1023/2070] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:27:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:27:04.936345 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4efc03-3df4-4f62-a61b-524faf99d4b9 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:27:04.937815 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-8d4efc03-3df4-4f62-a61b-524faf99d4b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:27:04.950804 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4efc03-3df4-4f62-a61b-524faf99d4b9 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:27:04.956281 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1417c3c2-3e5f-4090-bccf-bee128d71f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:27:04.962408 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1417c3c2-3e5f-4090-bccf-bee128d71f21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:04.962408 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1417c3c2-3e5f-4090-bccf-bee128d71f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:04.962408 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1417c3c2-3e5f-4090-bccf-bee128d71f21 admin admin] PATH_INFO: `/v3/projects/63581b8c221a497897c537bd6e36b83e` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:04.962408 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1417c3c2-3e5f-4090-bccf-bee128d71f21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63581b8c221a497897c537bd6e36b83e)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:27:04.962408 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1417c3c2-3e5f-4090-bccf-bee128d71f21 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:27:05.100329 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-1417c3c2-3e5f-4090-bccf-bee128d71f21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63581b8c221a497897c537bd6e36b83e', 'request_id': 'req-1417c3c2-3e5f-4090-bccf-bee128d71f21'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:05.112350 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-1417c3c2-3e5f-4090-bccf-bee128d71f21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63581b8c221a497897c537bd6e36b83e', 'request_id': 'req-1417c3c2-3e5f-4090-bccf-bee128d71f21'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:05.220402 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-1417c3c2-3e5f-4090-bccf-bee128d71f21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63581b8c221a497897c537bd6e36b83e', 'request_id': 'req-1417c3c2-3e5f-4090-bccf-bee128d71f21'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:05.233686 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-1417c3c2-3e5f-4090-bccf-bee128d71f21 admin admin] The token cache is being invalidate because project 63581b8c221a497897c537bd6e36b83e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:27:05.233977 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-1417c3c2-3e5f-4090-bccf-bee128d71f21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:05.234636 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-1417c3c2-3e5f-4090-bccf-bee128d71f21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:05.237011 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1048/2071] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:27:04 2022] DELETE /identity/v3/projects/63581b8c221a497897c537bd6e36b83e => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:27:05.241105 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fbbf5b-a1b4-420d-acb6-51919f87a2f7 service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:27:05.251449 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fbbf5b-a1b4-420d-acb6-51919f87a2f7 service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:27:05.270836 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:27:05.309584 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5723c9ba-4311-4348-b057-165932fedd15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:27:05.310244 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5723c9ba-4311-4348-b057-165932fedd15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:05.310523 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5723c9ba-4311-4348-b057-165932fedd15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:05.310799 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5723c9ba-4311-4348-b057-165932fedd15 admin admin] PATH_INFO: `/v3/users/a9a21f29f43947b194c60d8640057ed0` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:05.311997 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5723c9ba-4311-4348-b057-165932fedd15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9a21f29f43947b194c60d8640057ed0)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:27:05.313935 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5723c9ba-4311-4348-b057-165932fedd15 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:27:05.567107 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-5723c9ba-4311-4348-b057-165932fedd15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9a21f29f43947b194c60d8640057ed0'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:05.580476 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-5723c9ba-4311-4348-b057-165932fedd15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9a21f29f43947b194c60d8640057ed0'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:05.581195 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-5723c9ba-4311-4348-b057-165932fedd15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9a21f29f43947b194c60d8640057ed0'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:05.588571 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-5723c9ba-4311-4348-b057-165932fedd15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9a21f29f43947b194c60d8640057ed0'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:05.589211 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-5723c9ba-4311-4348-b057-165932fedd15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9a21f29f43947b194c60d8640057ed0'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:05.695634 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1024/2072] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:27:05 2022] DELETE /identity/v3/users/a9a21f29f43947b194c60d8640057ed0 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:27:06.211094 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1417c3c2-3e5f-4090-bccf-bee128d71f21 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:27:06.212999 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-1417c3c2-3e5f-4090-bccf-bee128d71f21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:27:06.233889 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:27:06.254316 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1417c3c2-3e5f-4090-bccf-bee128d71f21 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:27:06.339499 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e007dd0-3dd4-459b-9ff3-2e2469810a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:27:06.340192 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e007dd0-3dd4-459b-9ff3-2e2469810a96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:06.340472 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e007dd0-3dd4-459b-9ff3-2e2469810a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:06.340746 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e007dd0-3dd4-459b-9ff3-2e2469810a96 admin admin] PATH_INFO: `/v3/projects/21adbe8d4e874587a366162cab416b82` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:06.341975 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e007dd0-3dd4-459b-9ff3-2e2469810a96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21adbe8d4e874587a366162cab416b82)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:27:06.344177 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e007dd0-3dd4-459b-9ff3-2e2469810a96 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:27:06.460942 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-8e007dd0-3dd4-459b-9ff3-2e2469810a96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21adbe8d4e874587a366162cab416b82', 'request_id': 'req-8e007dd0-3dd4-459b-9ff3-2e2469810a96'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:06.466527 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-8e007dd0-3dd4-459b-9ff3-2e2469810a96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21adbe8d4e874587a366162cab416b82', 'request_id': 'req-8e007dd0-3dd4-459b-9ff3-2e2469810a96'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:06.535684 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-8e007dd0-3dd4-459b-9ff3-2e2469810a96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21adbe8d4e874587a366162cab416b82', 'request_id': 'req-8e007dd0-3dd4-459b-9ff3-2e2469810a96'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:06.549514 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-8e007dd0-3dd4-459b-9ff3-2e2469810a96 admin admin] The token cache is being invalidate because project 21adbe8d4e874587a366162cab416b82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:27:06.549838 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-8e007dd0-3dd4-459b-9ff3-2e2469810a96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:06.550522 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-8e007dd0-3dd4-459b-9ff3-2e2469810a96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:06.552173 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1049/2073] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:27:06 2022] DELETE /identity/v3/projects/21adbe8d4e874587a366162cab416b82 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:27:06.556320 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5723c9ba-4311-4348-b057-165932fedd15 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:27:06.558441 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-5723c9ba-4311-4348-b057-165932fedd15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:27:06.574727 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5723c9ba-4311-4348-b057-165932fedd15 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:27:06.654913 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ae6071-de1a-4b89-ac72-26c568b6e8f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:27:06.655749 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ae6071-de1a-4b89-ac72-26c568b6e8f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:06.661674 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ae6071-de1a-4b89-ac72-26c568b6e8f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:06.661674 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ae6071-de1a-4b89-ac72-26c568b6e8f3 admin admin] PATH_INFO: `/v3/users/1dad2b75f33f4424a04b64563adffbb7` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:06.661674 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ae6071-de1a-4b89-ac72-26c568b6e8f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1dad2b75f33f4424a04b64563adffbb7)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:27:06.661674 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ae6071-de1a-4b89-ac72-26c568b6e8f3 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:27:06.821828 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:27:06.835755 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-16ae6071-de1a-4b89-ac72-26c568b6e8f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1dad2b75f33f4424a04b64563adffbb7'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:06.855163 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-16ae6071-de1a-4b89-ac72-26c568b6e8f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1dad2b75f33f4424a04b64563adffbb7'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:06.855963 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-16ae6071-de1a-4b89-ac72-26c568b6e8f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1dad2b75f33f4424a04b64563adffbb7'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:06.870051 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-16ae6071-de1a-4b89-ac72-26c568b6e8f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1dad2b75f33f4424a04b64563adffbb7'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:06.871240 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-16ae6071-de1a-4b89-ac72-26c568b6e8f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1dad2b75f33f4424a04b64563adffbb7'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:06.953393 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1025/2074] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:27:06 2022] DELETE /identity/v3/users/1dad2b75f33f4424a04b64563adffbb7 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:27:07.465928 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e007dd0-3dd4-459b-9ff3-2e2469810a96 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:27:07.467527 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-8e007dd0-3dd4-459b-9ff3-2e2469810a96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:27:07.489183 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e007dd0-3dd4-459b-9ff3-2e2469810a96 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:27:07.552309 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47fddfc-a703-4518-b0a9-dda8a8e12a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:27:07.552964 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47fddfc-a703-4518-b0a9-dda8a8e12a4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:07.553250 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47fddfc-a703-4518-b0a9-dda8a8e12a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:07.553527 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47fddfc-a703-4518-b0a9-dda8a8e12a4e admin admin] PATH_INFO: `/v3/projects/a14dbec5278e433dbcbea9d68deb7a92` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:07.554743 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47fddfc-a703-4518-b0a9-dda8a8e12a4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a14dbec5278e433dbcbea9d68deb7a92)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:27:07.556753 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47fddfc-a703-4518-b0a9-dda8a8e12a4e admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:27:07.685644 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:27:07.697428 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-a47fddfc-a703-4518-b0a9-dda8a8e12a4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a14dbec5278e433dbcbea9d68deb7a92', 'request_id': 'req-a47fddfc-a703-4518-b0a9-dda8a8e12a4e'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:07.701552 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-a47fddfc-a703-4518-b0a9-dda8a8e12a4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a14dbec5278e433dbcbea9d68deb7a92', 'request_id': 'req-a47fddfc-a703-4518-b0a9-dda8a8e12a4e'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:07.775284 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-a47fddfc-a703-4518-b0a9-dda8a8e12a4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a14dbec5278e433dbcbea9d68deb7a92', 'request_id': 'req-a47fddfc-a703-4518-b0a9-dda8a8e12a4e'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:07.797731 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-a47fddfc-a703-4518-b0a9-dda8a8e12a4e admin admin] The token cache is being invalidate because project a14dbec5278e433dbcbea9d68deb7a92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:27:07.798084 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-a47fddfc-a703-4518-b0a9-dda8a8e12a4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:07.799108 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-a47fddfc-a703-4518-b0a9-dda8a8e12a4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:07.801113 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1050/2075] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:27:07 2022] DELETE /identity/v3/projects/a14dbec5278e433dbcbea9d68deb7a92 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:27:07.807284 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b9b586-919e-4110-9b9c-10ea0ad993e8 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:07.808798 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b9b586-919e-4110-9b9c-10ea0ad993e8 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:07.809088 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b9b586-919e-4110-9b9c-10ea0ad993e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:07.830957 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-05b9b586-919e-4110-9b9c-10ea0ad993e8 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:27:07.832230 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-05b9b586-919e-4110-9b9c-10ea0ad993e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:27:07.934887 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1026/2076] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:27:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:27:07.943673 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9800dff-f090-4851-b77d-6fd6fc6f957b None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:07.944238 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9800dff-f090-4851-b77d-6fd6fc6f957b None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:07.944667 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9800dff-f090-4851-b77d-6fd6fc6f957b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:07.967491 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-a9800dff-f090-4851-b77d-6fd6fc6f957b None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:27:07.972420 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-a9800dff-f090-4851-b77d-6fd6fc6f957b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:27:07.980432 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1051/2077] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:27:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:27:07.984958 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b9b586-919e-4110-9b9c-10ea0ad993e8 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:27:07.995527 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b9b586-919e-4110-9b9c-10ea0ad993e8 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:27:08.001009 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5e4307-bac8-4a48-b114-a6ff0f9be6ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:27:08.010283 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5e4307-bac8-4a48-b114-a6ff0f9be6ee admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:08.010283 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5e4307-bac8-4a48-b114-a6ff0f9be6ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:08.010283 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5e4307-bac8-4a48-b114-a6ff0f9be6ee admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:08.010283 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5e4307-bac8-4a48-b114-a6ff0f9be6ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:27:08.010283 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5e4307-bac8-4a48-b114-a6ff0f9be6ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:27:08.010283 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5e4307-bac8-4a48-b114-a6ff0f9be6ee admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:27:08.019032 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1027/2078] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Aug 4 21:27:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:27:08.024285 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9800dff-f090-4851-b77d-6fd6fc6f957b None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:27:08.048104 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9800dff-f090-4851-b77d-6fd6fc6f957b None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:27:08.057875 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3809c48e-0363-40e5-9739-9065c3f2284a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:27:08.057875 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3809c48e-0363-40e5-9739-9065c3f2284a admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:08.057875 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3809c48e-0363-40e5-9739-9065c3f2284a admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:08.057875 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3809c48e-0363-40e5-9739-9065c3f2284a admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:08.059213 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3809c48e-0363-40e5-9739-9065c3f2284a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:27:08.062379 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3809c48e-0363-40e5-9739-9065c3f2284a admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:27:08.164486 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1052/2079] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:27:08 2022] POST /identity/v3/projects => generated 385 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:27:08.168900 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5e4307-bac8-4a48-b114-a6ff0f9be6ee admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:27:08.181318 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5e4307-bac8-4a48-b114-a6ff0f9be6ee admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:27:08.257948 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f25b3e-6d60-469f-8b42-4209cea78c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:27:08.258705 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f25b3e-6d60-469f-8b42-4209cea78c87 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:08.260131 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f25b3e-6d60-469f-8b42-4209cea78c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:08.260458 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f25b3e-6d60-469f-8b42-4209cea78c87 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:08.261188 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f25b3e-6d60-469f-8b42-4209cea78c87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:27:08.263135 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f25b3e-6d60-469f-8b42-4209cea78c87 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:27:08.410022 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1028/2080] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:27:08 2022] POST /identity/v3/users => generated 293 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:27:08.414508 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3809c48e-0363-40e5-9739-9065c3f2284a admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:27:08.425479 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3809c48e-0363-40e5-9739-9065c3f2284a admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:27:08.430087 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dc98a7-63f7-4ed0-96ea-570c36e73787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:27:08.430711 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dc98a7-63f7-4ed0-96ea-570c36e73787 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:08.431001 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dc98a7-63f7-4ed0-96ea-570c36e73787 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:08.431279 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dc98a7-63f7-4ed0-96ea-570c36e73787 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:08.432060 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dc98a7-63f7-4ed0-96ea-570c36e73787 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:27:08.433613 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dc98a7-63f7-4ed0-96ea-570c36e73787 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:27:08.446612 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1053/2081] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:27:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:27:08.451309 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f25b3e-6d60-469f-8b42-4209cea78c87 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:27:08.462609 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f25b3e-6d60-469f-8b42-4209cea78c87 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:27:08.467825 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2562e48-6fb9-41e8-bdd7-fdd7f225da1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:27:08.468584 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2562e48-6fb9-41e8-bdd7-fdd7f225da1d admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:08.468923 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2562e48-6fb9-41e8-bdd7-fdd7f225da1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:08.469269 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2562e48-6fb9-41e8-bdd7-fdd7f225da1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:08.470109 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2562e48-6fb9-41e8-bdd7-fdd7f225da1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:27:08.472065 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2562e48-6fb9-41e8-bdd7-fdd7f225da1d admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:27:08.490187 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1029/2082] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:27:08 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:27:08.491484 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dc98a7-63f7-4ed0-96ea-570c36e73787 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:27:08.503784 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dc98a7-63f7-4ed0-96ea-570c36e73787 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:27:08.509312 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f132445-802b-4bcd-8d10-e33b50fe448a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:27:08.510578 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f132445-802b-4bcd-8d10-e33b50fe448a admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:08.521942 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f132445-802b-4bcd-8d10-e33b50fe448a admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:08.521942 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f132445-802b-4bcd-8d10-e33b50fe448a admin admin] PATH_INFO: `/v3/projects/26a44e355dca42a8be51d449297baa85/users/9eb3df5e7c52403ba244c9b10602a35f/roles/8feebc60363c4feb870718b1721e5324` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:08.540315 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f132445-802b-4bcd-8d10-e33b50fe448a admin admin] RBAC: Authorizing `identity:create_grant(project_id=26a44e355dca42a8be51d449297baa85, user_id=9eb3df5e7c52403ba244c9b10602a35f, role_id=8feebc60363c4feb870718b1721e5324)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:27:08.545933 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f132445-802b-4bcd-8d10-e33b50fe448a admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:27:08.743601 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1054/2083] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:27:08 2022] PUT /identity/v3/projects/26a44e355dca42a8be51d449297baa85/users/9eb3df5e7c52403ba244c9b10602a35f/roles/8feebc60363c4feb870718b1721e5324 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:27:08.756369 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9dfa1f-e128-48d3-8261-0d3e6e42d63f None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:08.756654 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9dfa1f-e128-48d3-8261-0d3e6e42d63f None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:08.757006 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9dfa1f-e128-48d3-8261-0d3e6e42d63f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:08.793103 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-9a9dfa1f-e128-48d3-8261-0d3e6e42d63f None None] MFA Rules not processed for user `9eb3df5e7c52403ba244c9b10602a35f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:27:08.794344 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-9a9dfa1f-e128-48d3-8261-0d3e6e42d63f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:27:08.811249 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:27:08.889396 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1030/2084] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:27:08 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:27:08.894481 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f132445-802b-4bcd-8d10-e33b50fe448a admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:27:08.965883 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f132445-802b-4bcd-8d10-e33b50fe448a admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:27:08.971049 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a3068a-ded3-4117-8238-c5389e839d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:27:08.971752 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a3068a-ded3-4117-8238-c5389e839d3a admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:08.972042 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a3068a-ded3-4117-8238-c5389e839d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:08.972343 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a3068a-ded3-4117-8238-c5389e839d3a admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:08.973025 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a3068a-ded3-4117-8238-c5389e839d3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:27:08.975161 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a3068a-ded3-4117-8238-c5389e839d3a admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:27:09.061852 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1055/2085] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:27:08 2022] POST /identity/v3/projects => generated 385 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:27:09.067522 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9dfa1f-e128-48d3-8261-0d3e6e42d63f None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:27:09.078466 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9dfa1f-e128-48d3-8261-0d3e6e42d63f None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:27:09.155118 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cf1cdf-f197-4c1e-807d-c38920362050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:27:09.155118 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cf1cdf-f197-4c1e-807d-c38920362050 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:09.155118 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cf1cdf-f197-4c1e-807d-c38920362050 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:09.155590 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cf1cdf-f197-4c1e-807d-c38920362050 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:09.156204 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cf1cdf-f197-4c1e-807d-c38920362050 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:27:09.158805 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cf1cdf-f197-4c1e-807d-c38920362050 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:27:09.307333 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1031/2086] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:27:09 2022] POST /identity/v3/users => generated 293 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:27:09.311421 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a3068a-ded3-4117-8238-c5389e839d3a admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:27:09.322815 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a3068a-ded3-4117-8238-c5389e839d3a admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:27:09.327708 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3649f8c3-a0e5-4848-b3db-a6be0cc47125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:27:09.328324 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3649f8c3-a0e5-4848-b3db-a6be0cc47125 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:09.328630 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3649f8c3-a0e5-4848-b3db-a6be0cc47125 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:09.328902 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3649f8c3-a0e5-4848-b3db-a6be0cc47125 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:09.329596 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3649f8c3-a0e5-4848-b3db-a6be0cc47125 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:27:09.331248 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3649f8c3-a0e5-4848-b3db-a6be0cc47125 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:27:09.346433 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1056/2087] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:27:09 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:27:09.352830 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cf1cdf-f197-4c1e-807d-c38920362050 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:27:09.364306 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cf1cdf-f197-4c1e-807d-c38920362050 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:27:09.370278 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a51473-183a-4258-be60-293aeb4b21b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:27:09.371053 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a51473-183a-4258-be60-293aeb4b21b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:09.371371 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a51473-183a-4258-be60-293aeb4b21b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:09.371658 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a51473-183a-4258-be60-293aeb4b21b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:09.372432 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a51473-183a-4258-be60-293aeb4b21b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:27:09.374065 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a51473-183a-4258-be60-293aeb4b21b4 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:27:09.385280 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1032/2088] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:27:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:27:09.397852 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3649f8c3-a0e5-4848-b3db-a6be0cc47125 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:27:09.407758 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3649f8c3-a0e5-4848-b3db-a6be0cc47125 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:27:09.415168 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bdf5fb-65a9-4c20-9ede-39b6d0ddbaf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:27:09.429915 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bdf5fb-65a9-4c20-9ede-39b6d0ddbaf0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:09.430217 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bdf5fb-65a9-4c20-9ede-39b6d0ddbaf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:09.430589 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bdf5fb-65a9-4c20-9ede-39b6d0ddbaf0 admin admin] PATH_INFO: `/v3/projects/ec941b5547ac4b63bb70a571cd928e94/users/27b9999d2a5044c0a9326aceb5891d7c/roles/8feebc60363c4feb870718b1721e5324` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:09.446664 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bdf5fb-65a9-4c20-9ede-39b6d0ddbaf0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec941b5547ac4b63bb70a571cd928e94, user_id=27b9999d2a5044c0a9326aceb5891d7c, role_id=8feebc60363c4feb870718b1721e5324)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:27:09.449442 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bdf5fb-65a9-4c20-9ede-39b6d0ddbaf0 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:27:09.547555 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1057/2089] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:27:09 2022] PUT /identity/v3/projects/ec941b5547ac4b63bb70a571cd928e94/users/27b9999d2a5044c0a9326aceb5891d7c/roles/8feebc60363c4feb870718b1721e5324 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:27:09.619076 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc4441-4a14-4a28-ac54-b32590bfc757 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:09.619265 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc4441-4a14-4a28-ac54-b32590bfc757 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:09.619565 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc4441-4a14-4a28-ac54-b32590bfc757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:09.653265 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-37fc4441-4a14-4a28-ac54-b32590bfc757 None None] MFA Rules not processed for user `27b9999d2a5044c0a9326aceb5891d7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:27:09.654781 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-37fc4441-4a14-4a28-ac54-b32590bfc757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:27:09.758192 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1033/2090] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:27:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:27:09.762837 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bdf5fb-65a9-4c20-9ede-39b6d0ddbaf0 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:27:09.773807 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:DELETE=5,keystone:SELECT=90,keystone:INSERT=5 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:27:09.827385 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bdf5fb-65a9-4c20-9ede-39b6d0ddbaf0 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:27:09.832543 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa931c37-d5d5-40bd-99ec-71d27fbcd86d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:27:09.833154 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa931c37-d5d5-40bd-99ec-71d27fbcd86d admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:09.833433 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa931c37-d5d5-40bd-99ec-71d27fbcd86d admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:09.833732 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa931c37-d5d5-40bd-99ec-71d27fbcd86d admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:09.834410 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa931c37-d5d5-40bd-99ec-71d27fbcd86d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:27:09.836356 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa931c37-d5d5-40bd-99ec-71d27fbcd86d admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:27:09.937651 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1058/2091] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:27:09 2022] POST /identity/v3/projects => generated 383 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:27:09.942770 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fc4441-4a14-4a28-ac54-b32590bfc757 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:27:09.956701 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fc4441-4a14-4a28-ac54-b32590bfc757 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:27:10.042850 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2a0b01-2435-4391-809e-c066553f24e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:27:10.043523 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2a0b01-2435-4391-809e-c066553f24e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:10.043996 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2a0b01-2435-4391-809e-c066553f24e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:10.044279 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2a0b01-2435-4391-809e-c066553f24e6 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:10.045052 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2a0b01-2435-4391-809e-c066553f24e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:27:10.047263 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2a0b01-2435-4391-809e-c066553f24e6 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:27:10.057920 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:27:10.146874 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1034/2092] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:27:09 2022] POST /identity/v3/users => generated 298 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:27:10.148216 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa931c37-d5d5-40bd-99ec-71d27fbcd86d admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:27:10.159508 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa931c37-d5d5-40bd-99ec-71d27fbcd86d admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:27:10.164389 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2cfdd8-62da-4547-8ae9-a9dd9dfe76f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:27:10.165015 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2cfdd8-62da-4547-8ae9-a9dd9dfe76f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:10.165299 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2cfdd8-62da-4547-8ae9-a9dd9dfe76f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:10.165570 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2cfdd8-62da-4547-8ae9-a9dd9dfe76f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:10.166269 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2cfdd8-62da-4547-8ae9-a9dd9dfe76f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:27:10.167940 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2cfdd8-62da-4547-8ae9-a9dd9dfe76f6 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:27:10.179603 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1059/2093] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:27:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:27:10.184572 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2a0b01-2435-4391-809e-c066553f24e6 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:27:10.205855 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2a0b01-2435-4391-809e-c066553f24e6 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:27:10.211514 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc48576-b097-4399-b553-bd8aa4407c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:27:10.214346 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc48576-b097-4399-b553-bd8aa4407c2b admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:10.214346 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc48576-b097-4399-b553-bd8aa4407c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:10.214346 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc48576-b097-4399-b553-bd8aa4407c2b admin admin] PATH_INFO: `/v3/projects/b19ca1cb27404fcabccb06bc05164577/users/2a823327174f4979a6d5d808e1622e40/roles/a432dd4735944ecfab2fa5ce1705d356` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:10.232074 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc48576-b097-4399-b553-bd8aa4407c2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b19ca1cb27404fcabccb06bc05164577, user_id=2a823327174f4979a6d5d808e1622e40, role_id=a432dd4735944ecfab2fa5ce1705d356)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:27:10.234139 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc48576-b097-4399-b553-bd8aa4407c2b admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:27:10.297006 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1035/2094] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:27:10 2022] PUT /identity/v3/projects/b19ca1cb27404fcabccb06bc05164577/users/2a823327174f4979a6d5d808e1622e40/roles/a432dd4735944ecfab2fa5ce1705d356 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:27:10.301854 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2cfdd8-62da-4547-8ae9-a9dd9dfe76f6 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:27:10.312449 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2cfdd8-62da-4547-8ae9-a9dd9dfe76f6 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:27:10.387019 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad57446-d49e-4393-8c9d-be2f48a7c687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:27:10.388154 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad57446-d49e-4393-8c9d-be2f48a7c687 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:10.388840 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad57446-d49e-4393-8c9d-be2f48a7c687 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:10.389315 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad57446-d49e-4393-8c9d-be2f48a7c687 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:10.390527 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad57446-d49e-4393-8c9d-be2f48a7c687 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:27:10.393551 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad57446-d49e-4393-8c9d-be2f48a7c687 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:27:10.430829 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1060/2095] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:27:10 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:27:10.443127 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc48576-b097-4399-b553-bd8aa4407c2b admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:27:10.465414 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc48576-b097-4399-b553-bd8aa4407c2b admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:27:10.474124 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc0316e-30fe-49c1-8a24-7a87fb67eca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:27:10.476132 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc0316e-30fe-49c1-8a24-7a87fb67eca9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:10.476426 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc0316e-30fe-49c1-8a24-7a87fb67eca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:10.476701 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc0316e-30fe-49c1-8a24-7a87fb67eca9 admin admin] PATH_INFO: `/v3/projects/b19ca1cb27404fcabccb06bc05164577/users/2a823327174f4979a6d5d808e1622e40/roles/a432dd4735944ecfab2fa5ce1705d356` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:10.484880 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc0316e-30fe-49c1-8a24-7a87fb67eca9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b19ca1cb27404fcabccb06bc05164577, user_id=2a823327174f4979a6d5d808e1622e40, role_id=a432dd4735944ecfab2fa5ce1705d356)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:27:10.489062 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc0316e-30fe-49c1-8a24-7a87fb67eca9 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:27:10.502244 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1036/2096] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:27:10 2022] PUT /identity/v3/projects/b19ca1cb27404fcabccb06bc05164577/users/2a823327174f4979a6d5d808e1622e40/roles/a432dd4735944ecfab2fa5ce1705d356 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:27:10.517190 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96554d41-a95c-4594-afcc-a2a05c22c5d3 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:10.518192 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96554d41-a95c-4594-afcc-a2a05c22c5d3 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:10.519544 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96554d41-a95c-4594-afcc-a2a05c22c5d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:10.552123 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-96554d41-a95c-4594-afcc-a2a05c22c5d3 None None] MFA Rules not processed for user `2a823327174f4979a6d5d808e1622e40`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:27:10.553386 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-96554d41-a95c-4594-afcc-a2a05c22c5d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:27:10.637515 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1061/2097] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:27:10 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:27:10.649228 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc0316e-30fe-49c1-8a24-7a87fb67eca9 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:27:10.651160 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-afc0316e-30fe-49c1-8a24-7a87fb67eca9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:27:10.714116 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc0316e-30fe-49c1-8a24-7a87fb67eca9 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:27:10.717981 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332dea46-e102-4eba-9180-37a301d7c3ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e31f2728056b460fa474478eb85ca1d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:27:10.718597 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332dea46-e102-4eba-9180-37a301d7c3ad service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:10.718902 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332dea46-e102-4eba-9180-37a301d7c3ad service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:10.719176 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332dea46-e102-4eba-9180-37a301d7c3ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:10.789729 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332dea46-e102-4eba-9180-37a301d7c3ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:27:10.791638 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332dea46-e102-4eba-9180-37a301d7c3ad service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:27:10.811428 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1037/2098] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:27:10 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:27:14.162132 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96554d41-a95c-4594-afcc-a2a05c22c5d3 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:27:14.171643 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:27:14.176004 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96554d41-a95c-4594-afcc-a2a05c22c5d3 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:27:14.183285 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca593301-ad35-4e50-adf7-53e1180c2848 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e31f2728056b460fa474478eb85ca1d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:27:14.184392 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca593301-ad35-4e50-adf7-53e1180c2848 service cinder] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:14.184562 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca593301-ad35-4e50-adf7-53e1180c2848 service cinder] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:14.184851 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca593301-ad35-4e50-adf7-53e1180c2848 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:14.257749 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca593301-ad35-4e50-adf7-53e1180c2848 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:27:14.259845 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca593301-ad35-4e50-adf7-53e1180c2848 service cinder] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:27:14.274766 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1062/2099] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:27:14 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:27:15.946828 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332dea46-e102-4eba-9180-37a301d7c3ad service cinder] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:27:15.955857 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332dea46-e102-4eba-9180-37a301d7c3ad service cinder] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:27:15.959645 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f40258-eef5-4e45-bfa7-5485114382cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e31f2728056b460fa474478eb85ca1d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:27:15.960282 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f40258-eef5-4e45-bfa7-5485114382cb service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:15.960569 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f40258-eef5-4e45-bfa7-5485114382cb service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:15.960849 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f40258-eef5-4e45-bfa7-5485114382cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:15.971187 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f40258-eef5-4e45-bfa7-5485114382cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:27:15.972702 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f40258-eef5-4e45-bfa7-5485114382cb service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:27:15.983422 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1038/2100] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:27:15 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:27:21.561880 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca593301-ad35-4e50-adf7-53e1180c2848 service cinder] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:27:21.655884 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca593301-ad35-4e50-adf7-53e1180c2848 service cinder] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:27:21.661264 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a05631-d3d4-4240-9616-c94385aed567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:27:21.661930 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a05631-d3d4-4240-9616-c94385aed567 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:21.662215 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a05631-d3d4-4240-9616-c94385aed567 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:21.662491 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a05631-d3d4-4240-9616-c94385aed567 admin admin] PATH_INFO: `/v3/users/9eb3df5e7c52403ba244c9b10602a35f` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:21.663739 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a05631-d3d4-4240-9616-c94385aed567 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9eb3df5e7c52403ba244c9b10602a35f)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:27:21.665701 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a05631-d3d4-4240-9616-c94385aed567 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:27:21.834635 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-04a05631-d3d4-4240-9616-c94385aed567 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9eb3df5e7c52403ba244c9b10602a35f'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:21.851453 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-04a05631-d3d4-4240-9616-c94385aed567 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9eb3df5e7c52403ba244c9b10602a35f'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:21.852223 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-04a05631-d3d4-4240-9616-c94385aed567 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9eb3df5e7c52403ba244c9b10602a35f'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:21.852856 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-04a05631-d3d4-4240-9616-c94385aed567 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9eb3df5e7c52403ba244c9b10602a35f'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:21.934469 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-04a05631-d3d4-4240-9616-c94385aed567 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9eb3df5e7c52403ba244c9b10602a35f'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:21.951067 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1063/2101] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:27:21 2022] DELETE /identity/v3/users/9eb3df5e7c52403ba244c9b10602a35f => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:27:21.958579 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95baddba-c9d3-43a2-9aaa-571751e6adb3 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:21.959124 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95baddba-c9d3-43a2-9aaa-571751e6adb3 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:21.959638 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95baddba-c9d3-43a2-9aaa-571751e6adb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:21.996872 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-95baddba-c9d3-43a2-9aaa-571751e6adb3 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:27:21.999378 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-95baddba-c9d3-43a2-9aaa-571751e6adb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:27:22.015755 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:27:22.073783 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1039/2102] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:27:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:27:22.084764 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a05631-d3d4-4240-9616-c94385aed567 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:27:22.087141 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-04a05631-d3d4-4240-9616-c94385aed567 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:27:22.122403 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:27:22.156112 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a05631-d3d4-4240-9616-c94385aed567 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:27:22.160028 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc75e85c-61a2-4287-9f86-838d9584bbf5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:27:22.160717 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc75e85c-61a2-4287-9f86-838d9584bbf5 service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:22.161151 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc75e85c-61a2-4287-9f86-838d9584bbf5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:22.161266 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc75e85c-61a2-4287-9f86-838d9584bbf5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:22.171306 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc75e85c-61a2-4287-9f86-838d9584bbf5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:27:22.172983 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc75e85c-61a2-4287-9f86-838d9584bbf5 service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:27:22.189904 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1064/2103] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:27:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:27:22.681712 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95baddba-c9d3-43a2-9aaa-571751e6adb3 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:27:22.683645 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-95baddba-c9d3-43a2-9aaa-571751e6adb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:27:22.712439 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95baddba-c9d3-43a2-9aaa-571751e6adb3 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:27:22.728164 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f01dfe-b38d-4d79-ba40-2de3ec19d3d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:27:22.732908 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f01dfe-b38d-4d79-ba40-2de3ec19d3d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:22.733197 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f01dfe-b38d-4d79-ba40-2de3ec19d3d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:22.733476 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f01dfe-b38d-4d79-ba40-2de3ec19d3d4 admin admin] PATH_INFO: `/v3/projects/26a44e355dca42a8be51d449297baa85` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:22.741018 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f01dfe-b38d-4d79-ba40-2de3ec19d3d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26a44e355dca42a8be51d449297baa85)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:27:22.743235 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f01dfe-b38d-4d79-ba40-2de3ec19d3d4 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:27:22.857791 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-f4f01dfe-b38d-4d79-ba40-2de3ec19d3d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26a44e355dca42a8be51d449297baa85', 'request_id': 'req-f4f01dfe-b38d-4d79-ba40-2de3ec19d3d4'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:22.864566 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-f4f01dfe-b38d-4d79-ba40-2de3ec19d3d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26a44e355dca42a8be51d449297baa85', 'request_id': 'req-f4f01dfe-b38d-4d79-ba40-2de3ec19d3d4'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:22.962771 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-f4f01dfe-b38d-4d79-ba40-2de3ec19d3d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26a44e355dca42a8be51d449297baa85', 'request_id': 'req-f4f01dfe-b38d-4d79-ba40-2de3ec19d3d4'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:22.976581 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-f4f01dfe-b38d-4d79-ba40-2de3ec19d3d4 admin admin] The token cache is being invalidate because project 26a44e355dca42a8be51d449297baa85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:27:22.976925 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-f4f01dfe-b38d-4d79-ba40-2de3ec19d3d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:22.977597 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-f4f01dfe-b38d-4d79-ba40-2de3ec19d3d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:22.982613 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1040/2104] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:27:22 2022] DELETE /identity/v3/projects/26a44e355dca42a8be51d449297baa85 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:27:22.987360 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc75e85c-61a2-4287-9f86-838d9584bbf5 service neutron] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:27:23.004274 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc75e85c-61a2-4287-9f86-838d9584bbf5 service neutron] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:27:23.082246 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efd1240-5ea3-4312-92ad-109c47069ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:27:23.082805 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efd1240-5ea3-4312-92ad-109c47069ac9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:23.083081 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efd1240-5ea3-4312-92ad-109c47069ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:23.083392 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efd1240-5ea3-4312-92ad-109c47069ac9 admin admin] PATH_INFO: `/v3/users/27b9999d2a5044c0a9326aceb5891d7c` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:23.084549 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efd1240-5ea3-4312-92ad-109c47069ac9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27b9999d2a5044c0a9326aceb5891d7c)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:27:23.086461 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efd1240-5ea3-4312-92ad-109c47069ac9 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:27:23.304469 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-7efd1240-5ea3-4312-92ad-109c47069ac9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27b9999d2a5044c0a9326aceb5891d7c'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:23.317177 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-7efd1240-5ea3-4312-92ad-109c47069ac9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27b9999d2a5044c0a9326aceb5891d7c'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:23.323298 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-7efd1240-5ea3-4312-92ad-109c47069ac9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27b9999d2a5044c0a9326aceb5891d7c'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:23.323298 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-7efd1240-5ea3-4312-92ad-109c47069ac9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27b9999d2a5044c0a9326aceb5891d7c'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:23.402688 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-7efd1240-5ea3-4312-92ad-109c47069ac9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27b9999d2a5044c0a9326aceb5891d7c'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:23.419395 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1065/2105] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:27:22 2022] DELETE /identity/v3/users/27b9999d2a5044c0a9326aceb5891d7c => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:27:23.525137 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c6008e-2cd6-4fd8-9947-8b0341b2bd73 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:23.525429 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c6008e-2cd6-4fd8-9947-8b0341b2bd73 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:23.525711 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c6008e-2cd6-4fd8-9947-8b0341b2bd73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:23.550474 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-e3c6008e-2cd6-4fd8-9947-8b0341b2bd73 None None] MFA Rules not processed for user `ea2f8480bc7d44f6a51e4a1d875a1c7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:27:23.551746 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-e3c6008e-2cd6-4fd8-9947-8b0341b2bd73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:27:23.610585 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1041/2106] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:27:23 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:27:23.863802 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efd1240-5ea3-4312-92ad-109c47069ac9 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:27:23.866213 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-7efd1240-5ea3-4312-92ad-109c47069ac9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:27:23.929541 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:27:23.955101 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efd1240-5ea3-4312-92ad-109c47069ac9 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:27:23.960612 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6817ef6-cc09-43c1-9988-be14547fa682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:27:23.971984 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6817ef6-cc09-43c1-9988-be14547fa682 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:23.971984 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6817ef6-cc09-43c1-9988-be14547fa682 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:23.971984 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6817ef6-cc09-43c1-9988-be14547fa682 admin admin] PATH_INFO: `/v3/projects/ec941b5547ac4b63bb70a571cd928e94` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:23.971984 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6817ef6-cc09-43c1-9988-be14547fa682 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec941b5547ac4b63bb70a571cd928e94)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:27:23.971984 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6817ef6-cc09-43c1-9988-be14547fa682 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:27:24.076470 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-b6817ef6-cc09-43c1-9988-be14547fa682 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec941b5547ac4b63bb70a571cd928e94', 'request_id': 'req-b6817ef6-cc09-43c1-9988-be14547fa682'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:24.084495 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-b6817ef6-cc09-43c1-9988-be14547fa682 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec941b5547ac4b63bb70a571cd928e94', 'request_id': 'req-b6817ef6-cc09-43c1-9988-be14547fa682'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:24.180143 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-b6817ef6-cc09-43c1-9988-be14547fa682 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec941b5547ac4b63bb70a571cd928e94', 'request_id': 'req-b6817ef6-cc09-43c1-9988-be14547fa682'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:24.206563 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-b6817ef6-cc09-43c1-9988-be14547fa682 admin admin] The token cache is being invalidate because project ec941b5547ac4b63bb70a571cd928e94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:27:24.206849 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-b6817ef6-cc09-43c1-9988-be14547fa682 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:24.207620 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-b6817ef6-cc09-43c1-9988-be14547fa682 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:24.209410 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1066/2107] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:27:23 2022] DELETE /identity/v3/projects/ec941b5547ac4b63bb70a571cd928e94 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:27:24.213828 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c6008e-2cd6-4fd8-9947-8b0341b2bd73 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:27:24.225735 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c6008e-2cd6-4fd8-9947-8b0341b2bd73 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:27:24.273397 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:27:24.291702 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa05cb9f-3f2e-4b7d-86f3-15e52b1acb39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:27:24.294163 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa05cb9f-3f2e-4b7d-86f3-15e52b1acb39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:24.294163 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa05cb9f-3f2e-4b7d-86f3-15e52b1acb39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:24.294163 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa05cb9f-3f2e-4b7d-86f3-15e52b1acb39 admin admin] PATH_INFO: `/v3/users/2a823327174f4979a6d5d808e1622e40` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:24.294438 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa05cb9f-3f2e-4b7d-86f3-15e52b1acb39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a823327174f4979a6d5d808e1622e40)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:27:24.296306 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa05cb9f-3f2e-4b7d-86f3-15e52b1acb39 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:27:24.511480 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-fa05cb9f-3f2e-4b7d-86f3-15e52b1acb39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a823327174f4979a6d5d808e1622e40'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:24.525234 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-fa05cb9f-3f2e-4b7d-86f3-15e52b1acb39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a823327174f4979a6d5d808e1622e40'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:24.526075 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-fa05cb9f-3f2e-4b7d-86f3-15e52b1acb39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a823327174f4979a6d5d808e1622e40'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:24.533363 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-fa05cb9f-3f2e-4b7d-86f3-15e52b1acb39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a823327174f4979a6d5d808e1622e40'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:24.534060 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-fa05cb9f-3f2e-4b7d-86f3-15e52b1acb39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a823327174f4979a6d5d808e1622e40'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:24.632695 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1042/2108] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:27:24 2022] DELETE /identity/v3/users/2a823327174f4979a6d5d808e1622e40 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:27:25.125107 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6817ef6-cc09-43c1-9988-be14547fa682 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:27:25.127336 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-b6817ef6-cc09-43c1-9988-be14547fa682 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:27:25.177520 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6817ef6-cc09-43c1-9988-be14547fa682 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:27:25.247444 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bbc996-05f0-4bc6-9b95-1d35c2af27f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:27:25.248682 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bbc996-05f0-4bc6-9b95-1d35c2af27f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:25.248682 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bbc996-05f0-4bc6-9b95-1d35c2af27f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:25.248682 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bbc996-05f0-4bc6-9b95-1d35c2af27f9 admin admin] PATH_INFO: `/v3/projects/b19ca1cb27404fcabccb06bc05164577` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:25.249932 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bbc996-05f0-4bc6-9b95-1d35c2af27f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b19ca1cb27404fcabccb06bc05164577)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:27:25.253288 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bbc996-05f0-4bc6-9b95-1d35c2af27f9 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:27:25.394400 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-76bbc996-05f0-4bc6-9b95-1d35c2af27f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b19ca1cb27404fcabccb06bc05164577', 'request_id': 'req-76bbc996-05f0-4bc6-9b95-1d35c2af27f9'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:25.400731 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-76bbc996-05f0-4bc6-9b95-1d35c2af27f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b19ca1cb27404fcabccb06bc05164577', 'request_id': 'req-76bbc996-05f0-4bc6-9b95-1d35c2af27f9'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:25.511726 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-76bbc996-05f0-4bc6-9b95-1d35c2af27f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b19ca1cb27404fcabccb06bc05164577', 'request_id': 'req-76bbc996-05f0-4bc6-9b95-1d35c2af27f9'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:25.526816 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-76bbc996-05f0-4bc6-9b95-1d35c2af27f9 admin admin] The token cache is being invalidate because project b19ca1cb27404fcabccb06bc05164577 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:27:25.527111 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-76bbc996-05f0-4bc6-9b95-1d35c2af27f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:25.527795 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-76bbc996-05f0-4bc6-9b95-1d35c2af27f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:25.529337 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1067/2109] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:27:25 2022] DELETE /identity/v3/projects/b19ca1cb27404fcabccb06bc05164577 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:27:25.534178 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec135ad-ea46-4eb6-81ea-dd25f2e199f4 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:25.534690 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec135ad-ea46-4eb6-81ea-dd25f2e199f4 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:25.534781 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec135ad-ea46-4eb6-81ea-dd25f2e199f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:25.556879 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-1ec135ad-ea46-4eb6-81ea-dd25f2e199f4 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:27:25.558135 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-1ec135ad-ea46-4eb6-81ea-dd25f2e199f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:27:25.630536 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1043/2110] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:27:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:27:25.639341 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7094e8e8-f13a-4712-84a6-0527620dac76 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:25.639699 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7094e8e8-f13a-4712-84a6-0527620dac76 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:25.639997 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7094e8e8-f13a-4712-84a6-0527620dac76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:25.648875 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:27:25.660326 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-7094e8e8-f13a-4712-84a6-0527620dac76 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:27:25.664228 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-7094e8e8-f13a-4712-84a6-0527620dac76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:27:25.669982 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1068/2111] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:27:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:27:25.674850 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec135ad-ea46-4eb6-81ea-dd25f2e199f4 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:27:25.685294 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec135ad-ea46-4eb6-81ea-dd25f2e199f4 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:27:25.691294 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa921f3-c217-42aa-b946-a77393b6b0bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:27:25.691952 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa921f3-c217-42aa-b946-a77393b6b0bb admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:25.692230 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa921f3-c217-42aa-b946-a77393b6b0bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:25.692503 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa921f3-c217-42aa-b946-a77393b6b0bb admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:25.693247 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa921f3-c217-42aa-b946-a77393b6b0bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:27:25.693607 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa921f3-c217-42aa-b946-a77393b6b0bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:27:25.695193 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa921f3-c217-42aa-b946-a77393b6b0bb admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:27:25.710180 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1044/2112] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Aug 4 21:27:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:27:25.714834 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7094e8e8-f13a-4712-84a6-0527620dac76 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:27:25.727669 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7094e8e8-f13a-4712-84a6-0527620dac76 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:27:25.733575 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506fd432-006b-4ffb-a0b0-70d5116e116c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:27:25.734211 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506fd432-006b-4ffb-a0b0-70d5116e116c admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:25.734488 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506fd432-006b-4ffb-a0b0-70d5116e116c admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:25.734759 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506fd432-006b-4ffb-a0b0-70d5116e116c admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:25.735503 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506fd432-006b-4ffb-a0b0-70d5116e116c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:27:25.737556 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506fd432-006b-4ffb-a0b0-70d5116e116c admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:27:25.889651 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1069/2113] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:27:25 2022] POST /identity/v3/projects => generated 373 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:27:25.894736 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa921f3-c217-42aa-b946-a77393b6b0bb admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:27:25.905578 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa921f3-c217-42aa-b946-a77393b6b0bb admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:27:25.958503 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:27:25.967596 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52694533-dd9a-4fe5-b63b-08b893f4e678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:27:25.968223 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52694533-dd9a-4fe5-b63b-08b893f4e678 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:25.968537 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52694533-dd9a-4fe5-b63b-08b893f4e678 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:25.968811 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52694533-dd9a-4fe5-b63b-08b893f4e678 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:25.969476 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52694533-dd9a-4fe5-b63b-08b893f4e678 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:27:25.972200 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52694533-dd9a-4fe5-b63b-08b893f4e678 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:27:26.049578 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1045/2114] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:27:25 2022] POST /identity/v3/users => generated 287 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:27:26.057050 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506fd432-006b-4ffb-a0b0-70d5116e116c admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:27:26.068432 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506fd432-006b-4ffb-a0b0-70d5116e116c admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:27:26.073358 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71170940-3552-4ed6-80ef-0491bfdea9f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:27:26.073983 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71170940-3552-4ed6-80ef-0491bfdea9f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:26.074269 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71170940-3552-4ed6-80ef-0491bfdea9f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:26.074538 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71170940-3552-4ed6-80ef-0491bfdea9f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:26.075238 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71170940-3552-4ed6-80ef-0491bfdea9f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:27:26.076884 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71170940-3552-4ed6-80ef-0491bfdea9f4 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:27:26.088898 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1070/2115] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:27:26 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:27:26.093200 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52694533-dd9a-4fe5-b63b-08b893f4e678 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:27:26.104697 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52694533-dd9a-4fe5-b63b-08b893f4e678 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:27:26.109958 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e8e432-1522-400e-852b-01a2858ddf1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:27:26.110734 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e8e432-1522-400e-852b-01a2858ddf1a admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:26.111696 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e8e432-1522-400e-852b-01a2858ddf1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:26.111974 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e8e432-1522-400e-852b-01a2858ddf1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:26.112675 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e8e432-1522-400e-852b-01a2858ddf1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:27:26.114294 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e8e432-1522-400e-852b-01a2858ddf1a admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:27:26.126175 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1046/2116] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:27:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:27:26.130486 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71170940-3552-4ed6-80ef-0491bfdea9f4 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:27:26.140692 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71170940-3552-4ed6-80ef-0491bfdea9f4 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:27:26.146269 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdd149f-d6c8-4670-8a5a-9dcf12bd4851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:27:26.146959 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdd149f-d6c8-4670-8a5a-9dcf12bd4851 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:26.147244 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdd149f-d6c8-4670-8a5a-9dcf12bd4851 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:26.147554 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdd149f-d6c8-4670-8a5a-9dcf12bd4851 admin admin] PATH_INFO: `/v3/projects/e734c237455f4d42b1e34eeb6ed38511/users/1039d83bfa58450ebe10e727f1dc0ea5/roles/8feebc60363c4feb870718b1721e5324` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:26.165366 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdd149f-d6c8-4670-8a5a-9dcf12bd4851 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e734c237455f4d42b1e34eeb6ed38511, user_id=1039d83bfa58450ebe10e727f1dc0ea5, role_id=8feebc60363c4feb870718b1721e5324)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:27:26.170616 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdd149f-d6c8-4670-8a5a-9dcf12bd4851 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:27:26.285456 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1071/2117] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:27:26 2022] PUT /identity/v3/projects/e734c237455f4d42b1e34eeb6ed38511/users/1039d83bfa58450ebe10e727f1dc0ea5/roles/8feebc60363c4feb870718b1721e5324 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:27:26.290765 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57871af-d8ad-491d-8ced-240b0be661d0 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:26.291052 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57871af-d8ad-491d-8ced-240b0be661d0 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:26.291379 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57871af-d8ad-491d-8ced-240b0be661d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:26.316085 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-c57871af-d8ad-491d-8ced-240b0be661d0 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:27:26.317437 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-c57871af-d8ad-491d-8ced-240b0be661d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:27:26.388337 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1047/2118] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:27:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:27:26.398175 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdd149f-d6c8-4670-8a5a-9dcf12bd4851 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:27:26.399979 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-2cdd149f-d6c8-4670-8a5a-9dcf12bd4851 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:27:26.462128 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdd149f-d6c8-4670-8a5a-9dcf12bd4851 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:27:26.465728 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544aee7b-7f04-40ce-8a62-c21091339012 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:27:26.466346 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544aee7b-7f04-40ce-8a62-c21091339012 service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:26.466626 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544aee7b-7f04-40ce-8a62-c21091339012 service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:26.466890 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544aee7b-7f04-40ce-8a62-c21091339012 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:26.477591 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544aee7b-7f04-40ce-8a62-c21091339012 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:27:26.479508 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544aee7b-7f04-40ce-8a62-c21091339012 service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:27:26.494483 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1072/2119] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:27:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:27:27.835677 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434c8ef5-99e0-42e6-9bac-dc1461d0d886 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:27.836507 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434c8ef5-99e0-42e6-9bac-dc1461d0d886 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:27.837069 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434c8ef5-99e0-42e6-9bac-dc1461d0d886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:27.891908 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-434c8ef5-99e0-42e6-9bac-dc1461d0d886 None None] MFA Rules not processed for user `3d5f53509b2f42269cd26e77c79ad8be`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:27:27.896242 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-434c8ef5-99e0-42e6-9bac-dc1461d0d886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:27:27.954180 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1048/2120] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:27:27 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:27:28.393049 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c256316-ad4b-467b-a3fa-bdf8a5da61e9 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:28.397814 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c256316-ad4b-467b-a3fa-bdf8a5da61e9 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:28.398339 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c256316-ad4b-467b-a3fa-bdf8a5da61e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:28.433361 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-1c256316-ad4b-467b-a3fa-bdf8a5da61e9 None None] MFA Rules not processed for user `1039d83bfa58450ebe10e727f1dc0ea5`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:27:28.434885 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-1c256316-ad4b-467b-a3fa-bdf8a5da61e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:27:28.527774 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1073/2121] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:27:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:27:28.539704 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434c8ef5-99e0-42e6-9bac-dc1461d0d886 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:27:28.541851 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-434c8ef5-99e0-42e6-9bac-dc1461d0d886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:27:28.637187 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434c8ef5-99e0-42e6-9bac-dc1461d0d886 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:27:28.641127 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc39255-e226-4ed3-af3d-d6f18b49384f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e31f2728056b460fa474478eb85ca1d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:27:28.641788 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc39255-e226-4ed3-af3d-d6f18b49384f service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:28.642649 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc39255-e226-4ed3-af3d-d6f18b49384f service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:28.642896 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc39255-e226-4ed3-af3d-d6f18b49384f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:28.653140 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc39255-e226-4ed3-af3d-d6f18b49384f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:27:28.656123 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc39255-e226-4ed3-af3d-d6f18b49384f service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:27:28.665851 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1049/2122] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:27:28 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:27:30.594469 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0525b0-ca2a-4525-a0b8-d34c4ff64e6a None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:30.595934 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0525b0-ca2a-4525-a0b8-d34c4ff64e6a None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:30.596386 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0525b0-ca2a-4525-a0b8-d34c4ff64e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:30.617578 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-eb0525b0-ca2a-4525-a0b8-d34c4ff64e6a None None] MFA Rules not processed for user `1039d83bfa58450ebe10e727f1dc0ea5`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:27:30.619179 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-eb0525b0-ca2a-4525-a0b8-d34c4ff64e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:27:30.626669 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1074/2123] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:27:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:27:38.661157 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=88,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:27:40.612716 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=86,keystone:INSERT=2 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:27:45.918117 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ada84b-2e4a-413c-a1c3-bf4629fca56f None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:45.918651 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ada84b-2e4a-413c-a1c3-bf4629fca56f None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:45.918651 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ada84b-2e4a-413c-a1c3-bf4629fca56f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:45.941094 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-88ada84b-2e4a-413c-a1c3-bf4629fca56f None None] MFA Rules not processed for user `23ea380385f649628cc27e3603bc700c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:27:45.942922 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-88ada84b-2e4a-413c-a1c3-bf4629fca56f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:27:46.013123 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1050/2124] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Aug 4 21:27:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:27:46.067435 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0525b0-ca2a-4525-a0b8-d34c4ff64e6a None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:27:46.075997 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-eb0525b0-ca2a-4525-a0b8-d34c4ff64e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:27:46.178344 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0525b0-ca2a-4525-a0b8-d34c4ff64e6a None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:27:46.185171 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa47788-b053-47ff-b16f-bc8f138edf61 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58935367c0ad445f8477cdac0df04d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:27:46.186481 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa47788-b053-47ff-b16f-bc8f138edf61 service swift] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:46.186930 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa47788-b053-47ff-b16f-bc8f138edf61 service swift] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:46.187257 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa47788-b053-47ff-b16f-bc8f138edf61 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:46.197006 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa47788-b053-47ff-b16f-bc8f138edf61 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:27:46.198626 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa47788-b053-47ff-b16f-bc8f138edf61 service swift] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:27:46.208013 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1075/2125] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Aug 4 21:27:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 21:27:47.052592 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ada84b-2e4a-413c-a1c3-bf4629fca56f None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:27:47.054247 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-88ada84b-2e4a-413c-a1c3-bf4629fca56f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:27:47.081313 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ada84b-2e4a-413c-a1c3-bf4629fca56f None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:27:47.120575 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b093a247-0574-4cea-bbdd-a90892828cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:27:47.120575 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b093a247-0574-4cea-bbdd-a90892828cfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:47.120575 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b093a247-0574-4cea-bbdd-a90892828cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:47.120575 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b093a247-0574-4cea-bbdd-a90892828cfa admin admin] PATH_INFO: `/v3/users/d895fcc24e4543aca8b23ae761630417` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:47.121315 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b093a247-0574-4cea-bbdd-a90892828cfa admin admin] RBAC: Authorizing `identity:delete_user(user_id=d895fcc24e4543aca8b23ae761630417)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:27:47.121315 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b093a247-0574-4cea-bbdd-a90892828cfa admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:27:47.454578 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa47788-b053-47ff-b16f-bc8f138edf61 service swift] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:27:47.460855 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-3fa47788-b053-47ff-b16f-bc8f138edf61 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:27:47.486766 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa47788-b053-47ff-b16f-bc8f138edf61 service swift] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:27:47.500147 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae12b1e9-524d-4206-848f-2d52e6fe08d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:27:47.500943 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae12b1e9-524d-4206-848f-2d52e6fe08d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:47.501229 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae12b1e9-524d-4206-848f-2d52e6fe08d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:47.501503 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae12b1e9-524d-4206-848f-2d52e6fe08d6 admin admin] PATH_INFO: `/v3/users/13846f6cdd664c7bbc75c2e5e7dcafd3` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:47.502799 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae12b1e9-524d-4206-848f-2d52e6fe08d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13846f6cdd664c7bbc75c2e5e7dcafd3)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:27:47.505231 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae12b1e9-524d-4206-848f-2d52e6fe08d6 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:27:47.580645 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-b093a247-0574-4cea-bbdd-a90892828cfa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd895fcc24e4543aca8b23ae761630417'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:47.594957 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-b093a247-0574-4cea-bbdd-a90892828cfa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd895fcc24e4543aca8b23ae761630417'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:47.595731 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-b093a247-0574-4cea-bbdd-a90892828cfa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd895fcc24e4543aca8b23ae761630417'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:47.606864 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-b093a247-0574-4cea-bbdd-a90892828cfa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd895fcc24e4543aca8b23ae761630417'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:47.608576 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-b093a247-0574-4cea-bbdd-a90892828cfa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd895fcc24e4543aca8b23ae761630417'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:47.676569 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1051/2126] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:27:47 2022] DELETE /identity/v3/users/d895fcc24e4543aca8b23ae761630417 => generated 0 bytes in 621 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:27:47.678793 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e70d801-75f5-48c7-8a9f-8d8538a0e7de None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:47.679436 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e70d801-75f5-48c7-8a9f-8d8538a0e7de None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:47.679832 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e70d801-75f5-48c7-8a9f-8d8538a0e7de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:47.702305 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-ae12b1e9-524d-4206-848f-2d52e6fe08d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13846f6cdd664c7bbc75c2e5e7dcafd3'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:47.728830 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-8e70d801-75f5-48c7-8a9f-8d8538a0e7de None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:27:47.732059 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-ae12b1e9-524d-4206-848f-2d52e6fe08d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13846f6cdd664c7bbc75c2e5e7dcafd3'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:47.732499 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-8e70d801-75f5-48c7-8a9f-8d8538a0e7de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:27:47.734360 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-ae12b1e9-524d-4206-848f-2d52e6fe08d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13846f6cdd664c7bbc75c2e5e7dcafd3'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:47.734986 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-ae12b1e9-524d-4206-848f-2d52e6fe08d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13846f6cdd664c7bbc75c2e5e7dcafd3'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:47.800645 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-ae12b1e9-524d-4206-848f-2d52e6fe08d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13846f6cdd664c7bbc75c2e5e7dcafd3'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:47.827191 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1076/2127] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:27:47 2022] DELETE /identity/v3/users/13846f6cdd664c7bbc75c2e5e7dcafd3 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:27:47.833081 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842880fc-3f97-48c8-9757-7b4e61424a6e None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:47.833526 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842880fc-3f97-48c8-9757-7b4e61424a6e None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:47.833647 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842880fc-3f97-48c8-9757-7b4e61424a6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:47.861791 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1052/2128] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:27:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:27:47.875443 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e70d801-75f5-48c7-8a9f-8d8538a0e7de None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:27:47.877055 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-8e70d801-75f5-48c7-8a9f-8d8538a0e7de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:27:47.893394 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-842880fc-3f97-48c8-9757-7b4e61424a6e None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:27:47.901885 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-842880fc-3f97-48c8-9757-7b4e61424a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:27:47.929469 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:27:48.020621 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e70d801-75f5-48c7-8a9f-8d8538a0e7de None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:27:48.024912 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1077/2129] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:27:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:27:48.039738 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842880fc-3f97-48c8-9757-7b4e61424a6e None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:27:48.043971 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:27:48.134595 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8090315-2079-4974-b73b-e6c05e4d2312 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:27:48.135306 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8090315-2079-4974-b73b-e6c05e4d2312 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:48.135591 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8090315-2079-4974-b73b-e6c05e4d2312 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:48.135825 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8090315-2079-4974-b73b-e6c05e4d2312 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:48.157452 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842880fc-3f97-48c8-9757-7b4e61424a6e None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:27:48.163226 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8090315-2079-4974-b73b-e6c05e4d2312 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:27:48.165944 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8090315-2079-4974-b73b-e6c05e4d2312 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:27:48.171939 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5850bb00-6aa3-49fb-a141-ec280eaceca7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:27:48.172592 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5850bb00-6aa3-49fb-a141-ec280eaceca7 service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:48.172933 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5850bb00-6aa3-49fb-a141-ec280eaceca7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:48.173204 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5850bb00-6aa3-49fb-a141-ec280eaceca7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:48.189727 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1053/2130] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:27:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:27:48.199186 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5850bb00-6aa3-49fb-a141-ec280eaceca7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:27:48.200696 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5850bb00-6aa3-49fb-a141-ec280eaceca7 service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:27:48.236973 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1078/2131] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:27:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:27:48.958899 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8090315-2079-4974-b73b-e6c05e4d2312 service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:27:48.960352 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-a8090315-2079-4974-b73b-e6c05e4d2312 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:27:48.994261 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8090315-2079-4974-b73b-e6c05e4d2312 service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:27:49.012984 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5850bb00-6aa3-49fb-a141-ec280eaceca7 service neutron] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:27:49.014512 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-5850bb00-6aa3-49fb-a141-ec280eaceca7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:27:49.029447 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f6e99b-b9b3-4b4d-a35a-ca94f888d8e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:27:49.030194 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f6e99b-b9b3-4b4d-a35a-ca94f888d8e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:49.030480 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f6e99b-b9b3-4b4d-a35a-ca94f888d8e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:49.030783 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f6e99b-b9b3-4b4d-a35a-ca94f888d8e5 admin admin] PATH_INFO: `/v3/projects/159668eb78014f06ae1940436ca305bf` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:49.032160 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f6e99b-b9b3-4b4d-a35a-ca94f888d8e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=159668eb78014f06ae1940436ca305bf)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:27:49.034135 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f6e99b-b9b3-4b4d-a35a-ca94f888d8e5 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:27:49.040907 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5850bb00-6aa3-49fb-a141-ec280eaceca7 service neutron] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:27:49.059502 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fee256-66c1-4e61-be10-454307dd4840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:27:49.060173 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fee256-66c1-4e61-be10-454307dd4840 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:49.060458 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fee256-66c1-4e61-be10-454307dd4840 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:49.060737 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fee256-66c1-4e61-be10-454307dd4840 admin admin] PATH_INFO: `/v3/projects/854bedcf8b864a369cae363a24282397` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:49.063237 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fee256-66c1-4e61-be10-454307dd4840 admin admin] RBAC: Authorizing `identity:delete_project(project_id=854bedcf8b864a369cae363a24282397)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:27:49.065143 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fee256-66c1-4e61-be10-454307dd4840 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:27:49.227346 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c7f6e99b-b9b3-4b4d-a35a-ca94f888d8e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '159668eb78014f06ae1940436ca305bf', 'request_id': 'req-c7f6e99b-b9b3-4b4d-a35a-ca94f888d8e5'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:49.235383 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c7f6e99b-b9b3-4b4d-a35a-ca94f888d8e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '159668eb78014f06ae1940436ca305bf', 'request_id': 'req-c7f6e99b-b9b3-4b4d-a35a-ca94f888d8e5'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:49.263025 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-e8fee256-66c1-4e61-be10-454307dd4840 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '854bedcf8b864a369cae363a24282397', 'request_id': 'req-e8fee256-66c1-4e61-be10-454307dd4840'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:49.271764 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-e8fee256-66c1-4e61-be10-454307dd4840 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '854bedcf8b864a369cae363a24282397', 'request_id': 'req-e8fee256-66c1-4e61-be10-454307dd4840'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:49.340058 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c7f6e99b-b9b3-4b4d-a35a-ca94f888d8e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '159668eb78014f06ae1940436ca305bf', 'request_id': 'req-c7f6e99b-b9b3-4b4d-a35a-ca94f888d8e5'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:49.353679 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c7f6e99b-b9b3-4b4d-a35a-ca94f888d8e5 admin admin] The token cache is being invalidate because project 159668eb78014f06ae1940436ca305bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:27:49.354374 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c7f6e99b-b9b3-4b4d-a35a-ca94f888d8e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:49.364502 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c7f6e99b-b9b3-4b4d-a35a-ca94f888d8e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:49.366446 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1054/2132] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:27:48 2022] DELETE /identity/v3/projects/159668eb78014f06ae1940436ca305bf => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:27:49.463778 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-e8fee256-66c1-4e61-be10-454307dd4840 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '854bedcf8b864a369cae363a24282397', 'request_id': 'req-e8fee256-66c1-4e61-be10-454307dd4840'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:49.488888 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-e8fee256-66c1-4e61-be10-454307dd4840 admin admin] The token cache is being invalidate because project 854bedcf8b864a369cae363a24282397 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:27:49.499774 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-e8fee256-66c1-4e61-be10-454307dd4840 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:49.500714 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-e8fee256-66c1-4e61-be10-454307dd4840 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:49.502640 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1079/2133] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:27:49 2022] DELETE /identity/v3/projects/854bedcf8b864a369cae363a24282397 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:27:49.507638 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f6e99b-b9b3-4b4d-a35a-ca94f888d8e5 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:27:49.512315 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-c7f6e99b-b9b3-4b4d-a35a-ca94f888d8e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:27:49.636365 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f6e99b-b9b3-4b4d-a35a-ca94f888d8e5 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:27:49.640831 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:27:49.752568 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2988da5-ff5a-4085-9dbb-8c23d31df920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:27:49.756474 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2988da5-ff5a-4085-9dbb-8c23d31df920 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:49.756474 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2988da5-ff5a-4085-9dbb-8c23d31df920 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:49.756474 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2988da5-ff5a-4085-9dbb-8c23d31df920 admin admin] PATH_INFO: `/v3/users/0c78d17525264066af7f9dfb4c3ea51a` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:49.756474 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2988da5-ff5a-4085-9dbb-8c23d31df920 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c78d17525264066af7f9dfb4c3ea51a)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:27:49.757728 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2988da5-ff5a-4085-9dbb-8c23d31df920 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:27:49.999224 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-f2988da5-ff5a-4085-9dbb-8c23d31df920 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c78d17525264066af7f9dfb4c3ea51a'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:50.025392 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-f2988da5-ff5a-4085-9dbb-8c23d31df920 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c78d17525264066af7f9dfb4c3ea51a'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:50.025851 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-f2988da5-ff5a-4085-9dbb-8c23d31df920 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c78d17525264066af7f9dfb4c3ea51a'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:50.036523 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-f2988da5-ff5a-4085-9dbb-8c23d31df920 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c78d17525264066af7f9dfb4c3ea51a'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:50.038586 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-f2988da5-ff5a-4085-9dbb-8c23d31df920 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c78d17525264066af7f9dfb4c3ea51a'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:50.119293 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1055/2134] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:27:49 2022] DELETE /identity/v3/users/0c78d17525264066af7f9dfb4c3ea51a => generated 0 bytes in 613 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:27:51.052901 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fee256-66c1-4e61-be10-454307dd4840 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:27:51.054685 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-e8fee256-66c1-4e61-be10-454307dd4840 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:27:51.100240 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fee256-66c1-4e61-be10-454307dd4840 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:27:51.234568 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff72a4d-20e9-4904-b977-cfc0de6329b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:27:51.236172 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff72a4d-20e9-4904-b977-cfc0de6329b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:51.236714 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff72a4d-20e9-4904-b977-cfc0de6329b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:51.237110 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff72a4d-20e9-4904-b977-cfc0de6329b0 admin admin] PATH_INFO: `/v3/projects/059492435a0045d19f7a838cb5d01e48` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:51.241115 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff72a4d-20e9-4904-b977-cfc0de6329b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=059492435a0045d19f7a838cb5d01e48)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:27:51.243210 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff72a4d-20e9-4904-b977-cfc0de6329b0 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:27:51.405571 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:27:51.450898 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-aff72a4d-20e9-4904-b977-cfc0de6329b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '059492435a0045d19f7a838cb5d01e48', 'request_id': 'req-aff72a4d-20e9-4904-b977-cfc0de6329b0'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:51.472032 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-aff72a4d-20e9-4904-b977-cfc0de6329b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '059492435a0045d19f7a838cb5d01e48', 'request_id': 'req-aff72a4d-20e9-4904-b977-cfc0de6329b0'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:51.557663 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-aff72a4d-20e9-4904-b977-cfc0de6329b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '059492435a0045d19f7a838cb5d01e48', 'request_id': 'req-aff72a4d-20e9-4904-b977-cfc0de6329b0'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:51.581633 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-aff72a4d-20e9-4904-b977-cfc0de6329b0 admin admin] The token cache is being invalidate because project 059492435a0045d19f7a838cb5d01e48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:27:51.581948 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-aff72a4d-20e9-4904-b977-cfc0de6329b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:51.587565 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-aff72a4d-20e9-4904-b977-cfc0de6329b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:51.591500 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1080/2135] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:27:51 2022] DELETE /identity/v3/projects/059492435a0045d19f7a838cb5d01e48 => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:27:51.596277 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a9a7c9-5243-41e0-9cb8-f5672e9fca56 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:51.596843 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a9a7c9-5243-41e0-9cb8-f5672e9fca56 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:51.596843 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a9a7c9-5243-41e0-9cb8-f5672e9fca56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:51.630611 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-d5a9a7c9-5243-41e0-9cb8-f5672e9fca56 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:27:51.631893 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-d5a9a7c9-5243-41e0-9cb8-f5672e9fca56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:27:51.716816 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1056/2136] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:27:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:27:51.724886 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c54e2d-a00c-4bc8-b900-003a6bc861a0 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:51.725195 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c54e2d-a00c-4bc8-b900-003a6bc861a0 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:51.725630 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c54e2d-a00c-4bc8-b900-003a6bc861a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:51.752071 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-b5c54e2d-a00c-4bc8-b900-003a6bc861a0 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:27:51.753294 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-b5c54e2d-a00c-4bc8-b900-003a6bc861a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:27:51.781494 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1081/2137] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:27:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:27:51.786776 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a9a7c9-5243-41e0-9cb8-f5672e9fca56 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:27:51.811486 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a9a7c9-5243-41e0-9cb8-f5672e9fca56 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:27:51.819373 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9269b60-a3dd-45c7-8c5a-0b6446324524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:27:51.820777 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9269b60-a3dd-45c7-8c5a-0b6446324524 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:51.821288 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9269b60-a3dd-45c7-8c5a-0b6446324524 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:51.821836 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9269b60-a3dd-45c7-8c5a-0b6446324524 admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:51.822828 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9269b60-a3dd-45c7-8c5a-0b6446324524 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:27:51.823745 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9269b60-a3dd-45c7-8c5a-0b6446324524 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:27:51.827534 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9269b60-a3dd-45c7-8c5a-0b6446324524 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:27:51.842512 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1057/2138] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Aug 4 21:27:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:27:51.848747 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c54e2d-a00c-4bc8-b900-003a6bc861a0 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:27:51.864901 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c54e2d-a00c-4bc8-b900-003a6bc861a0 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:27:51.871887 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405e2db3-4d05-4d73-a08b-a38e3e5db937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:27:51.872539 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405e2db3-4d05-4d73-a08b-a38e3e5db937 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:51.872824 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405e2db3-4d05-4d73-a08b-a38e3e5db937 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:51.873104 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405e2db3-4d05-4d73-a08b-a38e3e5db937 admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:51.874632 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405e2db3-4d05-4d73-a08b-a38e3e5db937 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:27:51.879559 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405e2db3-4d05-4d73-a08b-a38e3e5db937 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:27:51.965961 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1082/2139] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:27:51 2022] POST /identity/v3/projects => generated 381 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:27:51.970325 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9269b60-a3dd-45c7-8c5a-0b6446324524 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:27:51.986007 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9269b60-a3dd-45c7-8c5a-0b6446324524 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:27:51.999058 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:27:52.068011 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deba92d1-508c-472d-b6a5-b03a38130eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:27:52.068730 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deba92d1-508c-472d-b6a5-b03a38130eec admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:52.069086 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deba92d1-508c-472d-b6a5-b03a38130eec admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:52.069435 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deba92d1-508c-472d-b6a5-b03a38130eec admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:52.070204 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deba92d1-508c-472d-b6a5-b03a38130eec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:27:52.072413 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deba92d1-508c-472d-b6a5-b03a38130eec admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:27:52.181920 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1058/2140] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:27:51 2022] POST /identity/v3/users => generated 291 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:27:52.187050 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405e2db3-4d05-4d73-a08b-a38e3e5db937 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:27:52.208349 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405e2db3-4d05-4d73-a08b-a38e3e5db937 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:27:52.218125 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e70041-2043-4935-8098-626ed1f4ba6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:27:52.223373 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e70041-2043-4935-8098-626ed1f4ba6e admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:52.223373 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e70041-2043-4935-8098-626ed1f4ba6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:52.223373 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e70041-2043-4935-8098-626ed1f4ba6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:52.223373 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e70041-2043-4935-8098-626ed1f4ba6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:27:52.234301 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e70041-2043-4935-8098-626ed1f4ba6e admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:27:52.252654 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1083/2141] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:27:52 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:27:52.256375 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deba92d1-508c-472d-b6a5-b03a38130eec admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:27:52.271451 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deba92d1-508c-472d-b6a5-b03a38130eec admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:27:52.283708 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d9e83b-f112-4c15-9deb-971e668b897d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:27:52.284352 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d9e83b-f112-4c15-9deb-971e668b897d admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:52.284634 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d9e83b-f112-4c15-9deb-971e668b897d admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:52.284910 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d9e83b-f112-4c15-9deb-971e668b897d admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:52.286239 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d9e83b-f112-4c15-9deb-971e668b897d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:27:52.288230 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d9e83b-f112-4c15-9deb-971e668b897d admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:27:52.305277 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1059/2142] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:27:52 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:27:52.309121 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e70041-2043-4935-8098-626ed1f4ba6e admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:27:52.325672 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e70041-2043-4935-8098-626ed1f4ba6e admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:27:52.337054 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbbd1f8-d5f2-470f-a895-58a0682542b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:27:52.337757 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbbd1f8-d5f2-470f-a895-58a0682542b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:52.338044 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbbd1f8-d5f2-470f-a895-58a0682542b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:52.338364 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbbd1f8-d5f2-470f-a895-58a0682542b2 admin admin] PATH_INFO: `/v3/projects/62b8ccaa20e14f3caf2b1ee952d6ccbe/users/3a6a6185ae624848bfc731d0667bf753/roles/8feebc60363c4feb870718b1721e5324` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:52.362956 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbbd1f8-d5f2-470f-a895-58a0682542b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62b8ccaa20e14f3caf2b1ee952d6ccbe, user_id=3a6a6185ae624848bfc731d0667bf753, role_id=8feebc60363c4feb870718b1721e5324)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:27:52.366093 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbbd1f8-d5f2-470f-a895-58a0682542b2 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:27:52.452634 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1084/2143] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:27:52 2022] PUT /identity/v3/projects/62b8ccaa20e14f3caf2b1ee952d6ccbe/users/3a6a6185ae624848bfc731d0667bf753/roles/8feebc60363c4feb870718b1721e5324 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:27:52.479997 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4fad93-003e-4940-b8ca-67037845616e None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:52.480283 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4fad93-003e-4940-b8ca-67037845616e None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:52.480555 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4fad93-003e-4940-b8ca-67037845616e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:52.520638 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-0c4fad93-003e-4940-b8ca-67037845616e None None] MFA Rules not processed for user `3a6a6185ae624848bfc731d0667bf753`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:27:52.521879 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-0c4fad93-003e-4940-b8ca-67037845616e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:27:52.654895 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1060/2144] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:27:52 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:27:52.659767 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbbd1f8-d5f2-470f-a895-58a0682542b2 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:27:52.817329 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbbd1f8-d5f2-470f-a895-58a0682542b2 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:27:52.828468 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed885b02-ec93-418e-b7a7-3d6d80cf11aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:27:52.829495 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed885b02-ec93-418e-b7a7-3d6d80cf11aa admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:52.829933 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed885b02-ec93-418e-b7a7-3d6d80cf11aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:52.830211 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed885b02-ec93-418e-b7a7-3d6d80cf11aa admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:52.832969 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed885b02-ec93-418e-b7a7-3d6d80cf11aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:27:52.845742 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed885b02-ec93-418e-b7a7-3d6d80cf11aa admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:27:52.990403 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1085/2145] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:27:52 2022] POST /identity/v3/projects => generated 381 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:27:52.996153 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4fad93-003e-4940-b8ca-67037845616e None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:27:53.021305 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4fad93-003e-4940-b8ca-67037845616e None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:27:53.180593 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d1fdbe-9071-4e85-9c99-6f0649a2f2c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:27:53.181261 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d1fdbe-9071-4e85-9c99-6f0649a2f2c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:53.181541 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d1fdbe-9071-4e85-9c99-6f0649a2f2c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:53.181812 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d1fdbe-9071-4e85-9c99-6f0649a2f2c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:53.182495 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d1fdbe-9071-4e85-9c99-6f0649a2f2c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:27:53.196471 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d1fdbe-9071-4e85-9c99-6f0649a2f2c0 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:27:53.371728 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1061/2146] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:27:52 2022] POST /identity/v3/users => generated 297 bytes in 377 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:27:53.385449 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed885b02-ec93-418e-b7a7-3d6d80cf11aa admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:27:53.432962 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed885b02-ec93-418e-b7a7-3d6d80cf11aa admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:27:53.451631 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc51297-1659-45f2-89f5-d2c255b1b0c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:27:53.452924 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc51297-1659-45f2-89f5-d2c255b1b0c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:53.454870 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc51297-1659-45f2-89f5-d2c255b1b0c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:53.454870 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc51297-1659-45f2-89f5-d2c255b1b0c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:53.454870 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc51297-1659-45f2-89f5-d2c255b1b0c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:27:53.459348 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc51297-1659-45f2-89f5-d2c255b1b0c4 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:27:53.487641 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1086/2147] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:27:53 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:27:53.489352 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d1fdbe-9071-4e85-9c99-6f0649a2f2c0 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:27:53.505546 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d1fdbe-9071-4e85-9c99-6f0649a2f2c0 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:27:53.520115 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9f3c1a-114a-49a7-9209-cb83d3467967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:27:53.521092 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9f3c1a-114a-49a7-9209-cb83d3467967 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:53.521344 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9f3c1a-114a-49a7-9209-cb83d3467967 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:53.521723 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9f3c1a-114a-49a7-9209-cb83d3467967 admin admin] PATH_INFO: `/v3/projects/62980a64fc9848c29a32417cbd09ba5f/users/cefc5ad3b4d145049f29219753c2a7ad/roles/a432dd4735944ecfab2fa5ce1705d356` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:53.533220 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:27:53.541487 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9f3c1a-114a-49a7-9209-cb83d3467967 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62980a64fc9848c29a32417cbd09ba5f, user_id=cefc5ad3b4d145049f29219753c2a7ad, role_id=a432dd4735944ecfab2fa5ce1705d356)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:27:53.545939 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9f3c1a-114a-49a7-9209-cb83d3467967 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:27:53.628282 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1062/2148] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:27:53 2022] PUT /identity/v3/projects/62980a64fc9848c29a32417cbd09ba5f/users/cefc5ad3b4d145049f29219753c2a7ad/roles/a432dd4735944ecfab2fa5ce1705d356 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:27:53.632401 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc51297-1659-45f2-89f5-d2c255b1b0c4 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:27:53.647148 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc51297-1659-45f2-89f5-d2c255b1b0c4 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:27:53.692438 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:27:53.733697 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782bd76c-acfd-4a13-822d-3ee03afe955f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:27:53.734723 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782bd76c-acfd-4a13-822d-3ee03afe955f admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:53.735111 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782bd76c-acfd-4a13-822d-3ee03afe955f admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:53.735578 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782bd76c-acfd-4a13-822d-3ee03afe955f admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:53.736700 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782bd76c-acfd-4a13-822d-3ee03afe955f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:27:53.740427 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782bd76c-acfd-4a13-822d-3ee03afe955f admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:27:53.756103 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9f3c1a-114a-49a7-9209-cb83d3467967 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:27:53.757350 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1087/2149] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:27:53 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:27:53.768292 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9f3c1a-114a-49a7-9209-cb83d3467967 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:27:53.788066 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c532945-eab8-41b0-a788-f2aabf0f9345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:27:53.788941 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c532945-eab8-41b0-a788-f2aabf0f9345 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:53.789260 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c532945-eab8-41b0-a788-f2aabf0f9345 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:53.789572 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c532945-eab8-41b0-a788-f2aabf0f9345 admin admin] PATH_INFO: `/v3/projects/62980a64fc9848c29a32417cbd09ba5f/users/cefc5ad3b4d145049f29219753c2a7ad/roles/a432dd4735944ecfab2fa5ce1705d356` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:53.796558 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c532945-eab8-41b0-a788-f2aabf0f9345 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62980a64fc9848c29a32417cbd09ba5f, user_id=cefc5ad3b4d145049f29219753c2a7ad, role_id=a432dd4735944ecfab2fa5ce1705d356)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:27:53.801245 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c532945-eab8-41b0-a788-f2aabf0f9345 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:27:53.838031 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1063/2150] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:27:53 2022] PUT /identity/v3/projects/62980a64fc9848c29a32417cbd09ba5f/users/cefc5ad3b4d145049f29219753c2a7ad/roles/a432dd4735944ecfab2fa5ce1705d356 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:27:53.851404 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174533cb-deb7-4824-8862-2cf0d7bdc0a7 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:53.851702 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174533cb-deb7-4824-8862-2cf0d7bdc0a7 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:53.851987 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174533cb-deb7-4824-8862-2cf0d7bdc0a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:53.906911 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-174533cb-deb7-4824-8862-2cf0d7bdc0a7 None None] MFA Rules not processed for user `cefc5ad3b4d145049f29219753c2a7ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:27:53.911014 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-174533cb-deb7-4824-8862-2cf0d7bdc0a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:27:54.077279 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1088/2151] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:27:53 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:27:54.089282 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c532945-eab8-41b0-a788-f2aabf0f9345 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:27:54.092150 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-6c532945-eab8-41b0-a788-f2aabf0f9345 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:27:54.209026 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c532945-eab8-41b0-a788-f2aabf0f9345 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:27:54.214267 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e6fd92-61c7-40c8-a77b-3a56567e8d09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e31f2728056b460fa474478eb85ca1d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:27:54.214909 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e6fd92-61c7-40c8-a77b-3a56567e8d09 service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:54.215189 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e6fd92-61c7-40c8-a77b-3a56567e8d09 service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:54.215509 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e6fd92-61c7-40c8-a77b-3a56567e8d09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:54.226193 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e6fd92-61c7-40c8-a77b-3a56567e8d09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:27:54.227770 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e6fd92-61c7-40c8-a77b-3a56567e8d09 service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:27:54.240888 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1064/2152] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:27:54 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:27:54.438726 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174533cb-deb7-4824-8862-2cf0d7bdc0a7 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:27:54.451610 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174533cb-deb7-4824-8862-2cf0d7bdc0a7 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:27:54.456745 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306ebc13-f152-46cc-8faa-10379b2afcfa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e31f2728056b460fa474478eb85ca1d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:27:54.458190 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ebc13-f152-46cc-8faa-10379b2afcfa service cinder] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:54.458539 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ebc13-f152-46cc-8faa-10379b2afcfa service cinder] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:54.458837 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ebc13-f152-46cc-8faa-10379b2afcfa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:54.585391 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306ebc13-f152-46cc-8faa-10379b2afcfa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:27:54.586921 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306ebc13-f152-46cc-8faa-10379b2afcfa service cinder] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:27:54.598954 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1089/2153] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:27:54 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:27:58.328267 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e6fd92-61c7-40c8-a77b-3a56567e8d09 service cinder] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:27:58.444296 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e6fd92-61c7-40c8-a77b-3a56567e8d09 service cinder] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:27:58.454309 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eaa669-7b5c-4e70-bd7c-571a36365fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:27:58.454986 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eaa669-7b5c-4e70-bd7c-571a36365fe6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:58.455268 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eaa669-7b5c-4e70-bd7c-571a36365fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:58.455597 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eaa669-7b5c-4e70-bd7c-571a36365fe6 admin admin] PATH_INFO: `/v3/users/3a6a6185ae624848bfc731d0667bf753` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:58.457474 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2eaa669-7b5c-4e70-bd7c-571a36365fe6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a6a6185ae624848bfc731d0667bf753)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:27:58.459543 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2eaa669-7b5c-4e70-bd7c-571a36365fe6 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:27:58.998084 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-a2eaa669-7b5c-4e70-bd7c-571a36365fe6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a6a6185ae624848bfc731d0667bf753'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:59.029416 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-a2eaa669-7b5c-4e70-bd7c-571a36365fe6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a6a6185ae624848bfc731d0667bf753'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:59.029642 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-a2eaa669-7b5c-4e70-bd7c-571a36365fe6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a6a6185ae624848bfc731d0667bf753'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:59.038444 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-a2eaa669-7b5c-4e70-bd7c-571a36365fe6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a6a6185ae624848bfc731d0667bf753'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:59.039429 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-a2eaa669-7b5c-4e70-bd7c-571a36365fe6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a6a6185ae624848bfc731d0667bf753'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:27:59.046832 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:27:59.130885 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1065/2154] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:27:58 2022] DELETE /identity/v3/users/3a6a6185ae624848bfc731d0667bf753 => generated 0 bytes in 803 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:27:59.135700 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd178c5-29bf-47c6-8649-3e89ebf077eb None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:59.137293 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd178c5-29bf-47c6-8649-3e89ebf077eb None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:59.137293 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd178c5-29bf-47c6-8649-3e89ebf077eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:59.171919 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-acd178c5-29bf-47c6-8649-3e89ebf077eb None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:27:59.174000 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-acd178c5-29bf-47c6-8649-3e89ebf077eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:27:59.236990 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:27:59.259140 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1090/2155] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:27:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:27:59.274003 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eaa669-7b5c-4e70-bd7c-571a36365fe6 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:27:59.277332 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-a2eaa669-7b5c-4e70-bd7c-571a36365fe6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:27:59.367090 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eaa669-7b5c-4e70-bd7c-571a36365fe6 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:27:59.380607 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e3dc98-372e-4e49-8b97-72765c26465b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:27:59.382119 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e3dc98-372e-4e49-8b97-72765c26465b service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:27:59.382640 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e3dc98-372e-4e49-8b97-72765c26465b service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:27:59.383378 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e3dc98-372e-4e49-8b97-72765c26465b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:27:59.399684 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e3dc98-372e-4e49-8b97-72765c26465b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:27:59.401179 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e3dc98-372e-4e49-8b97-72765c26465b service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:27:59.419533 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1066/2156] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:27:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:27:59.988625 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd178c5-29bf-47c6-8649-3e89ebf077eb None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:27:59.990843 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-acd178c5-29bf-47c6-8649-3e89ebf077eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:28:00.013385 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd178c5-29bf-47c6-8649-3e89ebf077eb None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:28:00.025006 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4e5c04-b9cb-41f3-bb5c-53361418aa23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:28:00.025795 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4e5c04-b9cb-41f3-bb5c-53361418aa23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:00.026186 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4e5c04-b9cb-41f3-bb5c-53361418aa23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:00.026342 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4e5c04-b9cb-41f3-bb5c-53361418aa23 admin admin] PATH_INFO: `/v3/projects/62b8ccaa20e14f3caf2b1ee952d6ccbe` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:00.027645 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4e5c04-b9cb-41f3-bb5c-53361418aa23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62b8ccaa20e14f3caf2b1ee952d6ccbe)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:28:00.030483 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4e5c04-b9cb-41f3-bb5c-53361418aa23 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:28:00.056348 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759a7685-9927-4b18-8ed1-79c71d764604 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:00.056750 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759a7685-9927-4b18-8ed1-79c71d764604 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:00.057157 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759a7685-9927-4b18-8ed1-79c71d764604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:00.077001 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-759a7685-9927-4b18-8ed1-79c71d764604 None None] MFA Rules not processed for user `1039d83bfa58450ebe10e727f1dc0ea5`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:28:00.078194 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-759a7685-9927-4b18-8ed1-79c71d764604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:28:00.137596 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1067/2157] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:28:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:28:00.180819 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-9c4e5c04-b9cb-41f3-bb5c-53361418aa23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62b8ccaa20e14f3caf2b1ee952d6ccbe', 'request_id': 'req-9c4e5c04-b9cb-41f3-bb5c-53361418aa23'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:28:00.196197 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-9c4e5c04-b9cb-41f3-bb5c-53361418aa23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62b8ccaa20e14f3caf2b1ee952d6ccbe', 'request_id': 'req-9c4e5c04-b9cb-41f3-bb5c-53361418aa23'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:28:00.256621 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-9c4e5c04-b9cb-41f3-bb5c-53361418aa23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62b8ccaa20e14f3caf2b1ee952d6ccbe', 'request_id': 'req-9c4e5c04-b9cb-41f3-bb5c-53361418aa23'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:28:00.271862 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-9c4e5c04-b9cb-41f3-bb5c-53361418aa23 admin admin] The token cache is being invalidate because project 62b8ccaa20e14f3caf2b1ee952d6ccbe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:28:00.272246 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-9c4e5c04-b9cb-41f3-bb5c-53361418aa23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:28:00.272761 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-9c4e5c04-b9cb-41f3-bb5c-53361418aa23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:28:00.274335 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1091/2158] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:27:59 2022] DELETE /identity/v3/projects/62b8ccaa20e14f3caf2b1ee952d6ccbe => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:28:00.278788 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759a7685-9927-4b18-8ed1-79c71d764604 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:28:00.293586 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759a7685-9927-4b18-8ed1-79c71d764604 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:28:00.368068 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc80215-034e-41b5-beb1-30b7901af888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:28:00.368724 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc80215-034e-41b5-beb1-30b7901af888 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:00.369028 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc80215-034e-41b5-beb1-30b7901af888 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:00.369308 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc80215-034e-41b5-beb1-30b7901af888 admin admin] PATH_INFO: `/v3/users/cefc5ad3b4d145049f29219753c2a7ad` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:00.370590 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc80215-034e-41b5-beb1-30b7901af888 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cefc5ad3b4d145049f29219753c2a7ad)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:28:00.372658 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc80215-034e-41b5-beb1-30b7901af888 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:28:00.666894 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:28:00.668722 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-8fc80215-034e-41b5-beb1-30b7901af888 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cefc5ad3b4d145049f29219753c2a7ad'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:28:00.681174 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-8fc80215-034e-41b5-beb1-30b7901af888 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cefc5ad3b4d145049f29219753c2a7ad'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:28:00.681912 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-8fc80215-034e-41b5-beb1-30b7901af888 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cefc5ad3b4d145049f29219753c2a7ad'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:28:00.688891 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-8fc80215-034e-41b5-beb1-30b7901af888 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cefc5ad3b4d145049f29219753c2a7ad'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:28:00.689623 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-8fc80215-034e-41b5-beb1-30b7901af888 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cefc5ad3b4d145049f29219753c2a7ad'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:28:00.787905 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1068/2159] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:28:00 2022] DELETE /identity/v3/users/cefc5ad3b4d145049f29219753c2a7ad => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:28:01.331792 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4e5c04-b9cb-41f3-bb5c-53361418aa23 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:28:01.335171 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-9c4e5c04-b9cb-41f3-bb5c-53361418aa23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:28:01.365165 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4e5c04-b9cb-41f3-bb5c-53361418aa23 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:28:01.453369 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd34fb7-437e-448d-88d5-704fa06a712c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:28:01.454129 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd34fb7-437e-448d-88d5-704fa06a712c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:01.454492 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd34fb7-437e-448d-88d5-704fa06a712c admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:01.454904 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd34fb7-437e-448d-88d5-704fa06a712c admin admin] PATH_INFO: `/v3/projects/62980a64fc9848c29a32417cbd09ba5f` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:01.456378 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd34fb7-437e-448d-88d5-704fa06a712c admin admin] RBAC: Authorizing `identity:delete_project(project_id=62980a64fc9848c29a32417cbd09ba5f)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:28:01.458870 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd34fb7-437e-448d-88d5-704fa06a712c admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:28:01.593783 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-5cd34fb7-437e-448d-88d5-704fa06a712c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62980a64fc9848c29a32417cbd09ba5f', 'request_id': 'req-5cd34fb7-437e-448d-88d5-704fa06a712c'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:28:01.605213 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-5cd34fb7-437e-448d-88d5-704fa06a712c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62980a64fc9848c29a32417cbd09ba5f', 'request_id': 'req-5cd34fb7-437e-448d-88d5-704fa06a712c'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:28:01.705250 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-5cd34fb7-437e-448d-88d5-704fa06a712c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62980a64fc9848c29a32417cbd09ba5f', 'request_id': 'req-5cd34fb7-437e-448d-88d5-704fa06a712c'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:28:01.718644 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-5cd34fb7-437e-448d-88d5-704fa06a712c admin admin] The token cache is being invalidate because project 62980a64fc9848c29a32417cbd09ba5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:28:01.718946 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-5cd34fb7-437e-448d-88d5-704fa06a712c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:28:01.719614 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-5cd34fb7-437e-448d-88d5-704fa06a712c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:28:01.721165 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1092/2160] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:28:01 2022] DELETE /identity/v3/projects/62980a64fc9848c29a32417cbd09ba5f => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:28:01.726086 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d855806-6ef5-4ff9-8771-671eb98144d6 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:01.726784 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d855806-6ef5-4ff9-8771-671eb98144d6 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:01.726882 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d855806-6ef5-4ff9-8771-671eb98144d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:01.747838 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-1d855806-6ef5-4ff9-8771-671eb98144d6 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:28:01.749063 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-1d855806-6ef5-4ff9-8771-671eb98144d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:28:01.830276 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1069/2161] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:28:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:28:01.839701 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b758238-4d6d-4703-bd29-03bd7697aaa7 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:01.839881 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b758238-4d6d-4703-bd29-03bd7697aaa7 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:01.840856 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b758238-4d6d-4703-bd29-03bd7697aaa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:01.851770 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:28:01.865539 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-7b758238-4d6d-4703-bd29-03bd7697aaa7 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:28:01.866766 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-7b758238-4d6d-4703-bd29-03bd7697aaa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:28:01.875197 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1093/2162] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:28:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:28:01.880255 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d855806-6ef5-4ff9-8771-671eb98144d6 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:28:01.893223 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d855806-6ef5-4ff9-8771-671eb98144d6 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:28:01.900050 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de48775b-b9ba-40c5-9f4e-742234341374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:28:01.918941 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de48775b-b9ba-40c5-9f4e-742234341374 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:01.918941 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de48775b-b9ba-40c5-9f4e-742234341374 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:01.918941 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de48775b-b9ba-40c5-9f4e-742234341374 admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:01.918941 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de48775b-b9ba-40c5-9f4e-742234341374 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:28:01.918941 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de48775b-b9ba-40c5-9f4e-742234341374 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:28:01.918941 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de48775b-b9ba-40c5-9f4e-742234341374 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:28:01.919851 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1070/2163] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Aug 4 21:28:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:28:01.922155 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b758238-4d6d-4703-bd29-03bd7697aaa7 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:28:01.932498 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b758238-4d6d-4703-bd29-03bd7697aaa7 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:28:01.938287 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8694d529-cba6-44ff-9641-8a9d9cd3cb00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:28:01.939258 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8694d529-cba6-44ff-9641-8a9d9cd3cb00 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:01.939929 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8694d529-cba6-44ff-9641-8a9d9cd3cb00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:01.940736 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8694d529-cba6-44ff-9641-8a9d9cd3cb00 admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:01.943214 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8694d529-cba6-44ff-9641-8a9d9cd3cb00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:28:01.947972 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8694d529-cba6-44ff-9641-8a9d9cd3cb00 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:28:02.051733 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1094/2164] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:28:01 2022] POST /identity/v3/projects => generated 379 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:28:02.055965 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de48775b-b9ba-40c5-9f4e-742234341374 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:28:02.067133 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de48775b-b9ba-40c5-9f4e-742234341374 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:28:02.149030 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14335397-caa1-42d2-a5f2-c486e6339d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:28:02.149753 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14335397-caa1-42d2-a5f2-c486e6339d4a admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:02.150040 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14335397-caa1-42d2-a5f2-c486e6339d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:02.150312 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14335397-caa1-42d2-a5f2-c486e6339d4a admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:02.150980 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14335397-caa1-42d2-a5f2-c486e6339d4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:28:02.152945 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14335397-caa1-42d2-a5f2-c486e6339d4a admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:28:02.255089 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1071/2165] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:28:02 2022] POST /identity/v3/users => generated 290 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:28:02.259528 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8694d529-cba6-44ff-9641-8a9d9cd3cb00 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:28:02.270618 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8694d529-cba6-44ff-9641-8a9d9cd3cb00 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:28:02.275149 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d32c411-7227-4cf4-ac9e-e3edec21853f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:28:02.275828 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d32c411-7227-4cf4-ac9e-e3edec21853f admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:02.276112 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d32c411-7227-4cf4-ac9e-e3edec21853f admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:02.276385 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d32c411-7227-4cf4-ac9e-e3edec21853f admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:02.277082 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d32c411-7227-4cf4-ac9e-e3edec21853f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:28:02.278744 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d32c411-7227-4cf4-ac9e-e3edec21853f admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:28:02.289865 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1095/2166] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:28:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:28:02.299582 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14335397-caa1-42d2-a5f2-c486e6339d4a admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:28:02.311138 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14335397-caa1-42d2-a5f2-c486e6339d4a admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:28:02.316070 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac88f72-f405-4791-a029-d490a1b79311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:28:02.316912 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac88f72-f405-4791-a029-d490a1b79311 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:02.317092 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac88f72-f405-4791-a029-d490a1b79311 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:02.317370 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac88f72-f405-4791-a029-d490a1b79311 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:02.318069 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac88f72-f405-4791-a029-d490a1b79311 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:28:02.319766 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac88f72-f405-4791-a029-d490a1b79311 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:28:02.331236 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1072/2167] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:28:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:28:02.336350 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d32c411-7227-4cf4-ac9e-e3edec21853f admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:28:02.347019 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d32c411-7227-4cf4-ac9e-e3edec21853f admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:28:02.352748 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8b77f2-40af-46da-8c24-537d7de41dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:28:02.353491 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b77f2-40af-46da-8c24-537d7de41dac admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:02.353768 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b77f2-40af-46da-8c24-537d7de41dac admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:02.354037 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b77f2-40af-46da-8c24-537d7de41dac admin admin] PATH_INFO: `/v3/projects/72ad2541340e4643bd96be5e8f4fb44c/users/991a537a159c47abaa8b14ec2811a334/roles/8feebc60363c4feb870718b1721e5324` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:02.368335 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8b77f2-40af-46da-8c24-537d7de41dac admin admin] RBAC: Authorizing `identity:create_grant(project_id=72ad2541340e4643bd96be5e8f4fb44c, user_id=991a537a159c47abaa8b14ec2811a334, role_id=8feebc60363c4feb870718b1721e5324)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:28:02.371076 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8b77f2-40af-46da-8c24-537d7de41dac admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:28:02.437640 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1096/2168] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:28:02 2022] PUT /identity/v3/projects/72ad2541340e4643bd96be5e8f4fb44c/users/991a537a159c47abaa8b14ec2811a334/roles/8feebc60363c4feb870718b1721e5324 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:28:02.449681 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af7b09a-5294-4ffe-bf3b-220d73d74ecd None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:02.449973 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af7b09a-5294-4ffe-bf3b-220d73d74ecd None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:02.450254 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af7b09a-5294-4ffe-bf3b-220d73d74ecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:02.479089 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-1af7b09a-5294-4ffe-bf3b-220d73d74ecd None None] MFA Rules not processed for user `991a537a159c47abaa8b14ec2811a334`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:28:02.480386 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-1af7b09a-5294-4ffe-bf3b-220d73d74ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:28:02.497115 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:28:02.545549 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1073/2169] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:28:02 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:28:02.550332 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8b77f2-40af-46da-8c24-537d7de41dac admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:28:02.612713 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8b77f2-40af-46da-8c24-537d7de41dac admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:28:02.617505 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aa451d-57cb-45f8-ae1b-7bd1c7dbf57d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:28:02.618122 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aa451d-57cb-45f8-ae1b-7bd1c7dbf57d admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:02.618404 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aa451d-57cb-45f8-ae1b-7bd1c7dbf57d admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:02.618675 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aa451d-57cb-45f8-ae1b-7bd1c7dbf57d admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:02.619374 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01aa451d-57cb-45f8-ae1b-7bd1c7dbf57d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:28:02.621273 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01aa451d-57cb-45f8-ae1b-7bd1c7dbf57d admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:28:02.747082 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1097/2170] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:28:02 2022] POST /identity/v3/projects => generated 377 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:28:02.751254 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af7b09a-5294-4ffe-bf3b-220d73d74ecd None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:28:02.762532 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af7b09a-5294-4ffe-bf3b-220d73d74ecd None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:28:02.818934 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c4aa44-2eeb-4a27-bea7-727bcc7fcf70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:28:02.819587 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c4aa44-2eeb-4a27-bea7-727bcc7fcf70 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:02.819873 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c4aa44-2eeb-4a27-bea7-727bcc7fcf70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:02.820143 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c4aa44-2eeb-4a27-bea7-727bcc7fcf70 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:02.820904 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c4aa44-2eeb-4a27-bea7-727bcc7fcf70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:28:02.822943 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c4aa44-2eeb-4a27-bea7-727bcc7fcf70 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:28:02.898033 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1074/2171] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:28:02 2022] POST /identity/v3/users => generated 289 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:28:02.899561 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aa451d-57cb-45f8-ae1b-7bd1c7dbf57d admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:28:02.910362 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aa451d-57cb-45f8-ae1b-7bd1c7dbf57d admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:28:02.915167 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fcea53-f427-46d5-87ce-b06a4e60ba5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:28:02.915815 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fcea53-f427-46d5-87ce-b06a4e60ba5e admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:02.916093 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fcea53-f427-46d5-87ce-b06a4e60ba5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:02.916482 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fcea53-f427-46d5-87ce-b06a4e60ba5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:02.917182 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fcea53-f427-46d5-87ce-b06a4e60ba5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:28:02.918770 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fcea53-f427-46d5-87ce-b06a4e60ba5e admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:28:02.931730 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1098/2172] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:28:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:28:02.936043 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c4aa44-2eeb-4a27-bea7-727bcc7fcf70 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:28:02.947305 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c4aa44-2eeb-4a27-bea7-727bcc7fcf70 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:28:02.951988 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9a2dd0-aec1-42e4-9f20-f78893a468bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:28:02.952609 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9a2dd0-aec1-42e4-9f20-f78893a468bf admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:02.952894 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9a2dd0-aec1-42e4-9f20-f78893a468bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:02.953173 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9a2dd0-aec1-42e4-9f20-f78893a468bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:02.953875 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9a2dd0-aec1-42e4-9f20-f78893a468bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:28:02.955563 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9a2dd0-aec1-42e4-9f20-f78893a468bf admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:28:02.966042 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1075/2173] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:28:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:28:02.970306 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fcea53-f427-46d5-87ce-b06a4e60ba5e admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:28:02.980958 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fcea53-f427-46d5-87ce-b06a4e60ba5e admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:28:02.985527 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ca0587-6678-44fa-861a-fbd8ac71a11b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:28:02.986177 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ca0587-6678-44fa-861a-fbd8ac71a11b admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:02.986456 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ca0587-6678-44fa-861a-fbd8ac71a11b admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:02.986730 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ca0587-6678-44fa-861a-fbd8ac71a11b admin admin] PATH_INFO: `/v3/projects/f91df3fde9c64a74bfeac833054192ee/users/bed4cfd515854e2ba2371c335fdd65c7/roles/8feebc60363c4feb870718b1721e5324` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:03.001675 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ca0587-6678-44fa-861a-fbd8ac71a11b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f91df3fde9c64a74bfeac833054192ee, user_id=bed4cfd515854e2ba2371c335fdd65c7, role_id=8feebc60363c4feb870718b1721e5324)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:28:03.004893 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ca0587-6678-44fa-861a-fbd8ac71a11b admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:28:03.077360 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1099/2174] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:28:02 2022] PUT /identity/v3/projects/f91df3fde9c64a74bfeac833054192ee/users/bed4cfd515854e2ba2371c335fdd65c7/roles/8feebc60363c4feb870718b1721e5324 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:28:03.088324 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50aa9b9-5b0e-4ef2-a846-46d5d007ba9d None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:03.088774 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50aa9b9-5b0e-4ef2-a846-46d5d007ba9d None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:03.089060 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50aa9b9-5b0e-4ef2-a846-46d5d007ba9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:03.118795 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-e50aa9b9-5b0e-4ef2-a846-46d5d007ba9d None None] MFA Rules not processed for user `bed4cfd515854e2ba2371c335fdd65c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:28:03.120029 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-e50aa9b9-5b0e-4ef2-a846-46d5d007ba9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:28:03.184644 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1076/2175] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:28:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:28:03.189266 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ca0587-6678-44fa-861a-fbd8ac71a11b admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:28:03.254336 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:28:03.268828 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ca0587-6678-44fa-861a-fbd8ac71a11b admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:28:03.274173 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac9d068-4dc0-4bd0-80a5-44e7a7b8d912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:28:03.274802 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac9d068-4dc0-4bd0-80a5-44e7a7b8d912 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:03.275092 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac9d068-4dc0-4bd0-80a5-44e7a7b8d912 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:03.275400 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac9d068-4dc0-4bd0-80a5-44e7a7b8d912 admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:03.276368 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac9d068-4dc0-4bd0-80a5-44e7a7b8d912 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:28:03.278057 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac9d068-4dc0-4bd0-80a5-44e7a7b8d912 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:28:03.347582 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1100/2176] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:28:03 2022] POST /identity/v3/projects => generated 379 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:28:03.351886 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50aa9b9-5b0e-4ef2-a846-46d5d007ba9d None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:28:03.363162 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50aa9b9-5b0e-4ef2-a846-46d5d007ba9d None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:28:03.388134 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:28:03.420406 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c12d3db-71d2-4d09-8b12-42692b5c00f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:28:03.421025 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c12d3db-71d2-4d09-8b12-42692b5c00f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:03.421324 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c12d3db-71d2-4d09-8b12-42692b5c00f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:03.421578 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c12d3db-71d2-4d09-8b12-42692b5c00f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:03.422265 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c12d3db-71d2-4d09-8b12-42692b5c00f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:28:03.424394 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c12d3db-71d2-4d09-8b12-42692b5c00f1 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:28:03.488617 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1077/2177] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:28:03 2022] POST /identity/v3/users => generated 296 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:28:03.492610 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac9d068-4dc0-4bd0-80a5-44e7a7b8d912 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:28:03.503035 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac9d068-4dc0-4bd0-80a5-44e7a7b8d912 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:28:03.508286 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ce43b2-e525-4d1f-937b-291f7433c9bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:28:03.508836 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ce43b2-e525-4d1f-937b-291f7433c9bd admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:03.509118 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ce43b2-e525-4d1f-937b-291f7433c9bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:03.509395 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ce43b2-e525-4d1f-937b-291f7433c9bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:03.510094 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ce43b2-e525-4d1f-937b-291f7433c9bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:28:03.511915 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ce43b2-e525-4d1f-937b-291f7433c9bd admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:28:03.522842 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1101/2178] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:28:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:28:03.527172 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c12d3db-71d2-4d09-8b12-42692b5c00f1 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:28:03.537456 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c12d3db-71d2-4d09-8b12-42692b5c00f1 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:28:03.543615 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c673abcb-88c6-410e-9728-2a86ca147d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:28:03.544289 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c673abcb-88c6-410e-9728-2a86ca147d8e admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:03.544568 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c673abcb-88c6-410e-9728-2a86ca147d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:03.544841 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c673abcb-88c6-410e-9728-2a86ca147d8e admin admin] PATH_INFO: `/v3/projects/976f5f0b9e6d41d8a5c3f8429e14e7a2/users/a9338a31ce8d4be2851e2eaa88a901ef/roles/a432dd4735944ecfab2fa5ce1705d356` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:03.559748 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c673abcb-88c6-410e-9728-2a86ca147d8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=976f5f0b9e6d41d8a5c3f8429e14e7a2, user_id=a9338a31ce8d4be2851e2eaa88a901ef, role_id=a432dd4735944ecfab2fa5ce1705d356)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:28:03.562564 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c673abcb-88c6-410e-9728-2a86ca147d8e admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:28:03.643270 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1078/2179] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:28:03 2022] PUT /identity/v3/projects/976f5f0b9e6d41d8a5c3f8429e14e7a2/users/a9338a31ce8d4be2851e2eaa88a901ef/roles/a432dd4735944ecfab2fa5ce1705d356 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:28:03.647941 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ce43b2-e525-4d1f-937b-291f7433c9bd admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:28:03.658274 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ce43b2-e525-4d1f-937b-291f7433c9bd admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:28:03.717531 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905e4ea7-b3bb-4952-9a15-dd952a00bea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:28:03.718163 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e4ea7-b3bb-4952-9a15-dd952a00bea6 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:03.718452 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e4ea7-b3bb-4952-9a15-dd952a00bea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:03.718732 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e4ea7-b3bb-4952-9a15-dd952a00bea6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:03.719559 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905e4ea7-b3bb-4952-9a15-dd952a00bea6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:28:03.721188 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905e4ea7-b3bb-4952-9a15-dd952a00bea6 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:28:03.731355 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1102/2180] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:28:03 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:28:03.735554 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c673abcb-88c6-410e-9728-2a86ca147d8e admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:28:03.747050 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c673abcb-88c6-410e-9728-2a86ca147d8e admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:28:03.751738 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7b5c4d-fa47-44e9-ad7a-daf7647eb6b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:28:03.752397 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7b5c4d-fa47-44e9-ad7a-daf7647eb6b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:03.752677 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7b5c4d-fa47-44e9-ad7a-daf7647eb6b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:03.752947 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7b5c4d-fa47-44e9-ad7a-daf7647eb6b2 admin admin] PATH_INFO: `/v3/projects/976f5f0b9e6d41d8a5c3f8429e14e7a2/users/a9338a31ce8d4be2851e2eaa88a901ef/roles/a432dd4735944ecfab2fa5ce1705d356` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:03.755579 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7b5c4d-fa47-44e9-ad7a-daf7647eb6b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=976f5f0b9e6d41d8a5c3f8429e14e7a2, user_id=a9338a31ce8d4be2851e2eaa88a901ef, role_id=a432dd4735944ecfab2fa5ce1705d356)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:28:03.758224 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7b5c4d-fa47-44e9-ad7a-daf7647eb6b2 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:28:03.769663 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1079/2181] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:28:03 2022] PUT /identity/v3/projects/976f5f0b9e6d41d8a5c3f8429e14e7a2/users/a9338a31ce8d4be2851e2eaa88a901ef/roles/a432dd4735944ecfab2fa5ce1705d356 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:28:03.780986 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1787e14-614a-474c-9884-80d2713558a1 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:03.781272 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1787e14-614a-474c-9884-80d2713558a1 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:03.781548 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1787e14-614a-474c-9884-80d2713558a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:03.814893 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-e1787e14-614a-474c-9884-80d2713558a1 None None] MFA Rules not processed for user `a9338a31ce8d4be2851e2eaa88a901ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:28:03.817154 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-e1787e14-614a-474c-9884-80d2713558a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:28:03.890332 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1103/2182] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:28:03 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:28:03.901875 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7b5c4d-fa47-44e9-ad7a-daf7647eb6b2 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:28:03.903770 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-8a7b5c4d-fa47-44e9-ad7a-daf7647eb6b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:28:03.964070 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7b5c4d-fa47-44e9-ad7a-daf7647eb6b2 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:28:03.968557 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dd450c-8295-4c7d-b7d6-4d6cbf08df06 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e31f2728056b460fa474478eb85ca1d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:28:03.969311 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dd450c-8295-4c7d-b7d6-4d6cbf08df06 service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:03.969667 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dd450c-8295-4c7d-b7d6-4d6cbf08df06 service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:03.970013 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dd450c-8295-4c7d-b7d6-4d6cbf08df06 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:03.979861 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dd450c-8295-4c7d-b7d6-4d6cbf08df06 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:28:03.981343 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dd450c-8295-4c7d-b7d6-4d6cbf08df06 service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:28:03.992667 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1080/2183] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:28:03 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:28:04.164747 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1787e14-614a-474c-9884-80d2713558a1 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:28:04.167052 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-e1787e14-614a-474c-9884-80d2713558a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:28:04.180660 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1787e14-614a-474c-9884-80d2713558a1 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:28:04.183910 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56115b48-bdcb-435b-b27c-3257a9c23242 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e31f2728056b460fa474478eb85ca1d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:28:04.184532 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56115b48-bdcb-435b-b27c-3257a9c23242 service cinder] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:04.184820 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56115b48-bdcb-435b-b27c-3257a9c23242 service cinder] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:04.185095 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56115b48-bdcb-435b-b27c-3257a9c23242 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:04.235718 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:28:04.256723 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56115b48-bdcb-435b-b27c-3257a9c23242 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:28:04.257279 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56115b48-bdcb-435b-b27c-3257a9c23242 service cinder] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:28:04.267003 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1104/2184] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:28:04 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:28:06.480229 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4067f368-835d-42cb-91ed-7b31e373744b None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:06.481009 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4067f368-835d-42cb-91ed-7b31e373744b None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:06.481009 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4067f368-835d-42cb-91ed-7b31e373744b None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:06.482472 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1081/2185] 10.222.0.25 () {54 vars in 921 bytes} [Thu Aug 4 21:28:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:28:06.488537 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc39f741-072c-4582-a738-a489732b7724 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:06.489017 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc39f741-072c-4582-a738-a489732b7724 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:06.489098 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc39f741-072c-4582-a738-a489732b7724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:06.514501 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-fc39f741-072c-4582-a738-a489732b7724 None None] MFA Rules not processed for user `6a2ce1df87fa47e5be301916bf3245fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:28:06.515849 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-fc39f741-072c-4582-a738-a489732b7724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:28:06.618033 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1105/2186] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Aug 4 21:28:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:28:06.629984 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4067f368-835d-42cb-91ed-7b31e373744b None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:28:06.635523 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-4067f368-835d-42cb-91ed-7b31e373744b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:28:06.757998 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4067f368-835d-42cb-91ed-7b31e373744b None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:28:06.762430 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35177e34-c6f8-4598-9738-82597de348a1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6885c7d929f4f8e96d3890724c3b53f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:28:06.763294 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35177e34-c6f8-4598-9738-82597de348a1 service glance] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:06.763607 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35177e34-c6f8-4598-9738-82597de348a1 service glance] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:06.763883 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35177e34-c6f8-4598-9738-82597de348a1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:06.776893 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35177e34-c6f8-4598-9738-82597de348a1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:28:06.778417 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35177e34-c6f8-4598-9738-82597de348a1 service glance] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:28:06.791759 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1082/2187] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:28:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:28:07.022637 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1308f43b-efc6-42b0-906b-251a989024ff None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:07.023200 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1308f43b-efc6-42b0-906b-251a989024ff None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:07.023658 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1308f43b-efc6-42b0-906b-251a989024ff None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:07.025592 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1106/2188] 10.222.0.25 () {54 vars in 921 bytes} [Thu Aug 4 21:28:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:28:07.031073 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af35741-2c66-4944-9a9e-9a51e1a61615 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:07.031401 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af35741-2c66-4944-9a9e-9a51e1a61615 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:07.031682 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af35741-2c66-4944-9a9e-9a51e1a61615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:07.053604 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-7af35741-2c66-4944-9a9e-9a51e1a61615 None None] MFA Rules not processed for user `08fcb91ee54344b5bb3870c585e25fdc`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:28:07.053981 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-7af35741-2c66-4944-9a9e-9a51e1a61615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:28:07.084631 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:28:07.125031 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1083/2189] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Aug 4 21:28:07 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:28:07.137624 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1308f43b-efc6-42b0-906b-251a989024ff None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:28:07.139475 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-1308f43b-efc6-42b0-906b-251a989024ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:28:07.153942 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1308f43b-efc6-42b0-906b-251a989024ff None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:28:07.157624 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3266044e-ccf6-45c8-920e-341d05395f25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:28:07.158317 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3266044e-ccf6-45c8-920e-341d05395f25 service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:07.158672 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3266044e-ccf6-45c8-920e-341d05395f25 service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:07.159019 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3266044e-ccf6-45c8-920e-341d05395f25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:07.168266 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3266044e-ccf6-45c8-920e-341d05395f25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:28:07.169706 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3266044e-ccf6-45c8-920e-341d05395f25 service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:28:07.177719 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1107/2190] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:28:07 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:28:07.269021 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442e0385-2464-4887-9ba0-9f5080d392d0 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:07.269447 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442e0385-2464-4887-9ba0-9f5080d392d0 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:07.269779 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442e0385-2464-4887-9ba0-9f5080d392d0 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:07.271393 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1084/2191] 10.222.0.25 () {54 vars in 921 bytes} [Thu Aug 4 21:28:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:28:07.276460 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98784758-afe7-4669-8933-54c5c4da59ab None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:07.276868 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98784758-afe7-4669-8933-54c5c4da59ab None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:07.277193 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98784758-afe7-4669-8933-54c5c4da59ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:07.298443 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-98784758-afe7-4669-8933-54c5c4da59ab None None] MFA Rules not processed for user `6a2ce1df87fa47e5be301916bf3245fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:28:07.299838 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-98784758-afe7-4669-8933-54c5c4da59ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:28:07.310978 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1108/2192] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Aug 4 21:28:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:28:07.322054 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442e0385-2464-4887-9ba0-9f5080d392d0 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:28:07.333912 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442e0385-2464-4887-9ba0-9f5080d392d0 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:28:07.338530 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7e68fc-2796-4f1f-a782-775d0281bacf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:28:07.339168 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7e68fc-2796-4f1f-a782-775d0281bacf service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:07.339491 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7e68fc-2796-4f1f-a782-775d0281bacf service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:07.339775 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7e68fc-2796-4f1f-a782-775d0281bacf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:07.353054 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7e68fc-2796-4f1f-a782-775d0281bacf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:28:07.354686 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7e68fc-2796-4f1f-a782-775d0281bacf service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:28:07.367430 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1085/2193] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:28:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:28:08.657659 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98784758-afe7-4669-8933-54c5c4da59ab None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:28:08.725291 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98784758-afe7-4669-8933-54c5c4da59ab None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:28:08.729947 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b902e1-4e1c-4a7a-a133-7a945a2ee0f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:28:08.730597 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b902e1-4e1c-4a7a-a133-7a945a2ee0f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:08.730882 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b902e1-4e1c-4a7a-a133-7a945a2ee0f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:08.731152 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b902e1-4e1c-4a7a-a133-7a945a2ee0f0 admin admin] PATH_INFO: `/v3/users/991a537a159c47abaa8b14ec2811a334` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:08.732536 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b902e1-4e1c-4a7a-a133-7a945a2ee0f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=991a537a159c47abaa8b14ec2811a334)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:28:08.734514 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b902e1-4e1c-4a7a-a133-7a945a2ee0f0 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:28:08.917428 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-d0b902e1-4e1c-4a7a-a133-7a945a2ee0f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '991a537a159c47abaa8b14ec2811a334'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:28:08.924606 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:28:08.930880 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-d0b902e1-4e1c-4a7a-a133-7a945a2ee0f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '991a537a159c47abaa8b14ec2811a334'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:28:08.931746 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-d0b902e1-4e1c-4a7a-a133-7a945a2ee0f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '991a537a159c47abaa8b14ec2811a334'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:28:08.932365 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-d0b902e1-4e1c-4a7a-a133-7a945a2ee0f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '991a537a159c47abaa8b14ec2811a334'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:28:09.013581 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-d0b902e1-4e1c-4a7a-a133-7a945a2ee0f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '991a537a159c47abaa8b14ec2811a334'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:28:09.030391 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1109/2194] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:28:08 2022] DELETE /identity/v3/users/991a537a159c47abaa8b14ec2811a334 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:28:09.032308 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc26d4d-d244-4f8c-8677-9842e8692f28 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:09.032808 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc26d4d-d244-4f8c-8677-9842e8692f28 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:09.032974 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc26d4d-d244-4f8c-8677-9842e8692f28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:09.056405 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-3dc26d4d-d244-4f8c-8677-9842e8692f28 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:28:09.058020 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-3dc26d4d-d244-4f8c-8677-9842e8692f28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:28:09.132065 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1086/2195] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:28:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:28:09.141928 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b902e1-4e1c-4a7a-a133-7a945a2ee0f0 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:28:09.143470 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-d0b902e1-4e1c-4a7a-a133-7a945a2ee0f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:28:09.204063 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b902e1-4e1c-4a7a-a133-7a945a2ee0f0 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:28:09.207805 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb31eb4-773f-4ee0-890e-d7b22ff51ad1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:28:09.208438 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb31eb4-773f-4ee0-890e-d7b22ff51ad1 service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:09.208724 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb31eb4-773f-4ee0-890e-d7b22ff51ad1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:09.209003 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb31eb4-773f-4ee0-890e-d7b22ff51ad1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:09.221045 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb31eb4-773f-4ee0-890e-d7b22ff51ad1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:28:09.223160 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb31eb4-773f-4ee0-890e-d7b22ff51ad1 service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:28:09.236243 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1110/2196] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:28:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:28:09.719187 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc26d4d-d244-4f8c-8677-9842e8692f28 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:28:09.720706 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-3dc26d4d-d244-4f8c-8677-9842e8692f28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:28:09.733223 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc26d4d-d244-4f8c-8677-9842e8692f28 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:28:09.737908 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a551dbe-dfc8-4f9f-921d-69da98a6a562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:28:09.740797 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a551dbe-dfc8-4f9f-921d-69da98a6a562 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:09.740797 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a551dbe-dfc8-4f9f-921d-69da98a6a562 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:09.740797 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a551dbe-dfc8-4f9f-921d-69da98a6a562 admin admin] PATH_INFO: `/v3/projects/72ad2541340e4643bd96be5e8f4fb44c` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:09.740797 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a551dbe-dfc8-4f9f-921d-69da98a6a562 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72ad2541340e4643bd96be5e8f4fb44c)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:28:09.742362 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a551dbe-dfc8-4f9f-921d-69da98a6a562 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:28:09.829415 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb31eb4-773f-4ee0-890e-d7b22ff51ad1 service neutron] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:28:09.830890 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-2cb31eb4-773f-4ee0-890e-d7b22ff51ad1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:28:09.844694 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-0a551dbe-dfc8-4f9f-921d-69da98a6a562 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72ad2541340e4643bd96be5e8f4fb44c', 'request_id': 'req-0a551dbe-dfc8-4f9f-921d-69da98a6a562'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:28:09.851691 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-0a551dbe-dfc8-4f9f-921d-69da98a6a562 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72ad2541340e4643bd96be5e8f4fb44c', 'request_id': 'req-0a551dbe-dfc8-4f9f-921d-69da98a6a562'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:28:09.904357 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb31eb4-773f-4ee0-890e-d7b22ff51ad1 service neutron] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:28:09.934177 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-0a551dbe-dfc8-4f9f-921d-69da98a6a562 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72ad2541340e4643bd96be5e8f4fb44c', 'request_id': 'req-0a551dbe-dfc8-4f9f-921d-69da98a6a562'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:28:09.946918 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-0a551dbe-dfc8-4f9f-921d-69da98a6a562 admin admin] The token cache is being invalidate because project 72ad2541340e4643bd96be5e8f4fb44c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:28:09.947357 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-0a551dbe-dfc8-4f9f-921d-69da98a6a562 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:28:09.948014 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-0a551dbe-dfc8-4f9f-921d-69da98a6a562 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:28:09.949557 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1087/2197] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:28:09 2022] DELETE /identity/v3/projects/72ad2541340e4643bd96be5e8f4fb44c => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:28:09.953623 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a551dbe-dfc8-4f9f-921d-69da98a6a562 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:28:09.954987 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-0a551dbe-dfc8-4f9f-921d-69da98a6a562 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:28:09.972096 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600e7627-7f93-46ac-b65f-e84d17342ebd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67d44a0098ab4c36b14b36e9e59ae75c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:28:09.973213 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600e7627-7f93-46ac-b65f-e84d17342ebd service placement] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:09.973491 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600e7627-7f93-46ac-b65f-e84d17342ebd service placement] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:09.973758 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600e7627-7f93-46ac-b65f-e84d17342ebd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:09.975580 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-600e7627-7f93-46ac-b65f-e84d17342ebd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:28:09.987212 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600e7627-7f93-46ac-b65f-e84d17342ebd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:28:09.989601 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600e7627-7f93-46ac-b65f-e84d17342ebd service placement] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:28:09.999781 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1111/2198] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:28:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:28:10.006011 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:28:10.026404 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a551dbe-dfc8-4f9f-921d-69da98a6a562 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:28:10.031561 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f360c9d-8d8e-4657-89c3-9e2c2c29d431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:28:10.032408 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f360c9d-8d8e-4657-89c3-9e2c2c29d431 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:10.033305 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f360c9d-8d8e-4657-89c3-9e2c2c29d431 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:10.033753 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f360c9d-8d8e-4657-89c3-9e2c2c29d431 admin admin] PATH_INFO: `/v3/users/bed4cfd515854e2ba2371c335fdd65c7` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:10.035506 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f360c9d-8d8e-4657-89c3-9e2c2c29d431 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bed4cfd515854e2ba2371c335fdd65c7)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:28:10.038042 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f360c9d-8d8e-4657-89c3-9e2c2c29d431 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:28:10.240514 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-0f360c9d-8d8e-4657-89c3-9e2c2c29d431 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bed4cfd515854e2ba2371c335fdd65c7'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:28:10.253614 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-0f360c9d-8d8e-4657-89c3-9e2c2c29d431 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bed4cfd515854e2ba2371c335fdd65c7'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:28:10.254308 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-0f360c9d-8d8e-4657-89c3-9e2c2c29d431 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bed4cfd515854e2ba2371c335fdd65c7'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:28:10.266918 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-0f360c9d-8d8e-4657-89c3-9e2c2c29d431 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bed4cfd515854e2ba2371c335fdd65c7'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:28:10.267730 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-0f360c9d-8d8e-4657-89c3-9e2c2c29d431 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bed4cfd515854e2ba2371c335fdd65c7'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:28:10.363405 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1088/2199] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:28:09 2022] DELETE /identity/v3/users/bed4cfd515854e2ba2371c335fdd65c7 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:28:10.786890 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600e7627-7f93-46ac-b65f-e84d17342ebd service placement] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:28:10.798710 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600e7627-7f93-46ac-b65f-e84d17342ebd service placement] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:28:10.806555 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:28:10.865352 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b86b91-ecdb-4c1f-8f0f-cec2b2ff0c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:28:10.866037 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b86b91-ecdb-4c1f-8f0f-cec2b2ff0c32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:10.866294 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b86b91-ecdb-4c1f-8f0f-cec2b2ff0c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:10.866569 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b86b91-ecdb-4c1f-8f0f-cec2b2ff0c32 admin admin] PATH_INFO: `/v3/projects/f91df3fde9c64a74bfeac833054192ee` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:10.867816 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b86b91-ecdb-4c1f-8f0f-cec2b2ff0c32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f91df3fde9c64a74bfeac833054192ee)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:28:10.869823 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b86b91-ecdb-4c1f-8f0f-cec2b2ff0c32 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:28:10.974959 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-f3b86b91-ecdb-4c1f-8f0f-cec2b2ff0c32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f91df3fde9c64a74bfeac833054192ee', 'request_id': 'req-f3b86b91-ecdb-4c1f-8f0f-cec2b2ff0c32'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:28:10.980874 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-f3b86b91-ecdb-4c1f-8f0f-cec2b2ff0c32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f91df3fde9c64a74bfeac833054192ee', 'request_id': 'req-f3b86b91-ecdb-4c1f-8f0f-cec2b2ff0c32'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:28:11.066011 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-f3b86b91-ecdb-4c1f-8f0f-cec2b2ff0c32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f91df3fde9c64a74bfeac833054192ee', 'request_id': 'req-f3b86b91-ecdb-4c1f-8f0f-cec2b2ff0c32'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:28:11.079177 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-f3b86b91-ecdb-4c1f-8f0f-cec2b2ff0c32 admin admin] The token cache is being invalidate because project f91df3fde9c64a74bfeac833054192ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:28:11.079509 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-f3b86b91-ecdb-4c1f-8f0f-cec2b2ff0c32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:28:11.080462 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-f3b86b91-ecdb-4c1f-8f0f-cec2b2ff0c32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:28:11.082290 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1112/2200] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:28:10 2022] DELETE /identity/v3/projects/f91df3fde9c64a74bfeac833054192ee => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:28:11.086419 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f360c9d-8d8e-4657-89c3-9e2c2c29d431 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:28:11.088246 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-0f360c9d-8d8e-4657-89c3-9e2c2c29d431 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:28:11.102043 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f360c9d-8d8e-4657-89c3-9e2c2c29d431 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:28:11.180849 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7b5a1b-4115-4bd1-999a-f5855aac6a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:28:11.181676 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7b5a1b-4115-4bd1-999a-f5855aac6a37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:11.181972 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7b5a1b-4115-4bd1-999a-f5855aac6a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:11.182250 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7b5a1b-4115-4bd1-999a-f5855aac6a37 admin admin] PATH_INFO: `/v3/users/a9338a31ce8d4be2851e2eaa88a901ef` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:11.183581 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7b5a1b-4115-4bd1-999a-f5855aac6a37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9338a31ce8d4be2851e2eaa88a901ef)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:28:11.185605 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7b5a1b-4115-4bd1-999a-f5855aac6a37 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:28:11.390835 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-7a7b5a1b-4115-4bd1-999a-f5855aac6a37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9338a31ce8d4be2851e2eaa88a901ef'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:28:11.419431 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-7a7b5a1b-4115-4bd1-999a-f5855aac6a37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9338a31ce8d4be2851e2eaa88a901ef'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:28:11.420734 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-7a7b5a1b-4115-4bd1-999a-f5855aac6a37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9338a31ce8d4be2851e2eaa88a901ef'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:28:11.422792 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:28:11.430920 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-7a7b5a1b-4115-4bd1-999a-f5855aac6a37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9338a31ce8d4be2851e2eaa88a901ef'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:28:11.431747 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-7a7b5a1b-4115-4bd1-999a-f5855aac6a37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9338a31ce8d4be2851e2eaa88a901ef'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:28:11.518128 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1089/2201] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:28:11 2022] DELETE /identity/v3/users/a9338a31ce8d4be2851e2eaa88a901ef => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:28:12.044531 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b86b91-ecdb-4c1f-8f0f-cec2b2ff0c32 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:28:12.046074 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-f3b86b91-ecdb-4c1f-8f0f-cec2b2ff0c32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:28:12.060899 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b86b91-ecdb-4c1f-8f0f-cec2b2ff0c32 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:28:12.123247 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5c05d0-7831-47fd-aa17-3da621e7b7ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:28:12.124070 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5c05d0-7831-47fd-aa17-3da621e7b7ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:12.124256 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5c05d0-7831-47fd-aa17-3da621e7b7ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:12.124553 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5c05d0-7831-47fd-aa17-3da621e7b7ff admin admin] PATH_INFO: `/v3/projects/976f5f0b9e6d41d8a5c3f8429e14e7a2` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:12.125854 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5c05d0-7831-47fd-aa17-3da621e7b7ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=976f5f0b9e6d41d8a5c3f8429e14e7a2)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:28:12.127890 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5c05d0-7831-47fd-aa17-3da621e7b7ff admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:28:12.237635 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:28:12.245235 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-bd5c05d0-7831-47fd-aa17-3da621e7b7ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '976f5f0b9e6d41d8a5c3f8429e14e7a2', 'request_id': 'req-bd5c05d0-7831-47fd-aa17-3da621e7b7ff'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:28:12.251382 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-bd5c05d0-7831-47fd-aa17-3da621e7b7ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '976f5f0b9e6d41d8a5c3f8429e14e7a2', 'request_id': 'req-bd5c05d0-7831-47fd-aa17-3da621e7b7ff'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:28:12.307632 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ced282-7370-400b-83bb-5d287709d558 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:12.308090 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ced282-7370-400b-83bb-5d287709d558 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:12.308225 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ced282-7370-400b-83bb-5d287709d558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:12.323301 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-bd5c05d0-7831-47fd-aa17-3da621e7b7ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '976f5f0b9e6d41d8a5c3f8429e14e7a2', 'request_id': 'req-bd5c05d0-7831-47fd-aa17-3da621e7b7ff'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:28:12.328357 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-21ced282-7370-400b-83bb-5d287709d558 None None] MFA Rules not processed for user `23ea380385f649628cc27e3603bc700c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:28:12.329978 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-21ced282-7370-400b-83bb-5d287709d558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:28:12.336068 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-bd5c05d0-7831-47fd-aa17-3da621e7b7ff admin admin] The token cache is being invalidate because project 976f5f0b9e6d41d8a5c3f8429e14e7a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:28:12.336377 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-bd5c05d0-7831-47fd-aa17-3da621e7b7ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:28:12.337050 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-bd5c05d0-7831-47fd-aa17-3da621e7b7ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:28:12.338610 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1113/2202] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:28:12 2022] DELETE /identity/v3/projects/976f5f0b9e6d41d8a5c3f8429e14e7a2 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:28:12.345383 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f7357b-f812-447a-9ba7-910669aef404 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:12.345666 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f7357b-f812-447a-9ba7-910669aef404 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:12.345936 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f7357b-f812-447a-9ba7-910669aef404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:12.366094 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-e1f7357b-f812-447a-9ba7-910669aef404 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:28:12.367336 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-e1f7357b-f812-447a-9ba7-910669aef404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:28:12.389415 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1090/2203] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Aug 4 21:28:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:28:12.402513 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ced282-7370-400b-83bb-5d287709d558 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:28:12.403526 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-21ced282-7370-400b-83bb-5d287709d558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:28:12.437086 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1114/2204] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:28:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:28:12.445590 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21758648-93c5-43c4-879b-ab0003bf5a5d None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:12.445883 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21758648-93c5-43c4-879b-ab0003bf5a5d None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:12.446159 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21758648-93c5-43c4-879b-ab0003bf5a5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:12.466079 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-21758648-93c5-43c4-879b-ab0003bf5a5d None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:28:12.467376 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-21758648-93c5-43c4-879b-ab0003bf5a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:28:12.476870 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1115/2205] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:28:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:28:12.481180 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21758648-93c5-43c4-879b-ab0003bf5a5d None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:28:12.483022 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ced282-7370-400b-83bb-5d287709d558 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:28:12.488833 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291d4dd3-c4b7-44fe-aadf-d7d83f01eb9f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58935367c0ad445f8477cdac0df04d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:28:12.489015 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291d4dd3-c4b7-44fe-aadf-d7d83f01eb9f service swift] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:12.489330 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291d4dd3-c4b7-44fe-aadf-d7d83f01eb9f service swift] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:12.489611 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291d4dd3-c4b7-44fe-aadf-d7d83f01eb9f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:12.490589 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21758648-93c5-43c4-879b-ab0003bf5a5d None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:28:12.495367 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdb4e36-1121-407f-ba99-d26675ea63bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:28:12.496038 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb4e36-1121-407f-ba99-d26675ea63bc admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:12.496321 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb4e36-1121-407f-ba99-d26675ea63bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:12.496589 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb4e36-1121-407f-ba99-d26675ea63bc admin admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:12.497284 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdb4e36-1121-407f-ba99-d26675ea63bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:28:12.497583 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdb4e36-1121-407f-ba99-d26675ea63bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:28:12.498242 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291d4dd3-c4b7-44fe-aadf-d7d83f01eb9f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:28:12.499185 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdb4e36-1121-407f-ba99-d26675ea63bc admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:28:12.500062 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291d4dd3-c4b7-44fe-aadf-d7d83f01eb9f service swift] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:28:12.508392 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1091/2206] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Aug 4 21:28:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 04 21:28:12.514448 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1116/2207] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Aug 4 21:28:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:28:12.515407 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291d4dd3-c4b7-44fe-aadf-d7d83f01eb9f service swift] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:28:12.527823 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291d4dd3-c4b7-44fe-aadf-d7d83f01eb9f service swift] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:28:12.532894 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dcaad3-b0cb-45c0-8caa-7ac0b5c8e225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:28:12.533516 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dcaad3-b0cb-45c0-8caa-7ac0b5c8e225 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:12.533803 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dcaad3-b0cb-45c0-8caa-7ac0b5c8e225 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:12.534080 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dcaad3-b0cb-45c0-8caa-7ac0b5c8e225 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:12.534828 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dcaad3-b0cb-45c0-8caa-7ac0b5c8e225 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:28:12.537296 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dcaad3-b0cb-45c0-8caa-7ac0b5c8e225 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:28:12.675865 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1092/2208] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:28:12 2022] POST /identity/v3/projects => generated 375 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:28:12.678605 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdb4e36-1121-407f-ba99-d26675ea63bc admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:28:12.689121 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdb4e36-1121-407f-ba99-d26675ea63bc admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:28:12.793651 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e1c6b1-1591-49f9-8fb1-def54f83e6fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:28:12.795488 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e1c6b1-1591-49f9-8fb1-def54f83e6fd admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:12.796419 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e1c6b1-1591-49f9-8fb1-def54f83e6fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:12.798369 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e1c6b1-1591-49f9-8fb1-def54f83e6fd admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:12.799772 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e1c6b1-1591-49f9-8fb1-def54f83e6fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:28:12.822680 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e1c6b1-1591-49f9-8fb1-def54f83e6fd admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:28:12.944803 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1117/2209] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:28:12 2022] POST /identity/v3/users => generated 288 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:28:12.949413 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dcaad3-b0cb-45c0-8caa-7ac0b5c8e225 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:28:12.988594 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dcaad3-b0cb-45c0-8caa-7ac0b5c8e225 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:28:12.988594 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5639cd5c-b709-4a75-807b-e9bcb33c551d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:28:12.988594 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5639cd5c-b709-4a75-807b-e9bcb33c551d admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:12.988594 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5639cd5c-b709-4a75-807b-e9bcb33c551d admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:12.988998 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5639cd5c-b709-4a75-807b-e9bcb33c551d admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:12.988998 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5639cd5c-b709-4a75-807b-e9bcb33c551d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:28:12.990452 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5639cd5c-b709-4a75-807b-e9bcb33c551d admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:28:13.011577 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e1c6b1-1591-49f9-8fb1-def54f83e6fd admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:28:13.011860 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1093/2210] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:28:12 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:28:13.028852 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e1c6b1-1591-49f9-8fb1-def54f83e6fd admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:28:13.034027 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d518470e-c99d-4f5c-be4d-add34c6a37ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:28:13.034597 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d518470e-c99d-4f5c-be4d-add34c6a37ef admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:13.034954 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d518470e-c99d-4f5c-be4d-add34c6a37ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:13.035207 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d518470e-c99d-4f5c-be4d-add34c6a37ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:13.036224 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d518470e-c99d-4f5c-be4d-add34c6a37ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:28:13.039346 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d518470e-c99d-4f5c-be4d-add34c6a37ef admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:28:13.052672 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1118/2211] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:28:13 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:28:13.057329 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5639cd5c-b709-4a75-807b-e9bcb33c551d admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:28:13.074207 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5639cd5c-b709-4a75-807b-e9bcb33c551d admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:28:13.084760 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2641d12-c6dd-48b3-9900-2874604f8701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:28:13.086164 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2641d12-c6dd-48b3-9900-2874604f8701 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:13.086842 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2641d12-c6dd-48b3-9900-2874604f8701 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:13.087650 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2641d12-c6dd-48b3-9900-2874604f8701 admin admin] PATH_INFO: `/v3/projects/371e9c7913f04a539c2ebd2c54b4672a/users/00819e68d8044dc98d2ce2264abe3f95/roles/8feebc60363c4feb870718b1721e5324` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:13.119476 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2641d12-c6dd-48b3-9900-2874604f8701 admin admin] RBAC: Authorizing `identity:create_grant(project_id=371e9c7913f04a539c2ebd2c54b4672a, user_id=00819e68d8044dc98d2ce2264abe3f95, role_id=8feebc60363c4feb870718b1721e5324)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:28:13.122731 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2641d12-c6dd-48b3-9900-2874604f8701 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:28:13.203769 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1094/2212] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:28:13 2022] PUT /identity/v3/projects/371e9c7913f04a539c2ebd2c54b4672a/users/00819e68d8044dc98d2ce2264abe3f95/roles/8feebc60363c4feb870718b1721e5324 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:28:13.208887 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee76d43-b6e0-4eca-8396-c7108633bd28 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:13.209249 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee76d43-b6e0-4eca-8396-c7108633bd28 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:13.209605 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee76d43-b6e0-4eca-8396-c7108633bd28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:13.248165 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-8ee76d43-b6e0-4eca-8396-c7108633bd28 None None] MFA Rules not processed for user `00819e68d8044dc98d2ce2264abe3f95`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:28:13.249922 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-8ee76d43-b6e0-4eca-8396-c7108633bd28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:28:13.290892 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:28:13.372267 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1119/2213] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:28:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:28:13.379051 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2641d12-c6dd-48b3-9900-2874604f8701 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:28:13.381804 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-c2641d12-c6dd-48b3-9900-2874604f8701 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:28:13.408443 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:28:13.450668 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2641d12-c6dd-48b3-9900-2874604f8701 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:28:13.457519 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4c3029-413c-4aaa-9d27-29ccbfce285e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e31f2728056b460fa474478eb85ca1d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:28:13.458607 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4c3029-413c-4aaa-9d27-29ccbfce285e service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:13.459228 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4c3029-413c-4aaa-9d27-29ccbfce285e service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:13.459674 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4c3029-413c-4aaa-9d27-29ccbfce285e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:13.473965 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4c3029-413c-4aaa-9d27-29ccbfce285e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:28:13.475489 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4c3029-413c-4aaa-9d27-29ccbfce285e service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:28:13.485462 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1095/2214] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:28:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:28:17.879498 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee76d43-b6e0-4eca-8396-c7108633bd28 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:28:17.967673 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee76d43-b6e0-4eca-8396-c7108633bd28 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:28:17.973951 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdced2bf-e0c7-4deb-861a-68c18db47848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:28:17.974631 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdced2bf-e0c7-4deb-861a-68c18db47848 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:17.974919 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdced2bf-e0c7-4deb-861a-68c18db47848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:17.975250 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdced2bf-e0c7-4deb-861a-68c18db47848 admin admin] PATH_INFO: `/v3/users/00819e68d8044dc98d2ce2264abe3f95` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:17.976536 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdced2bf-e0c7-4deb-861a-68c18db47848 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00819e68d8044dc98d2ce2264abe3f95)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:28:17.979082 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdced2bf-e0c7-4deb-861a-68c18db47848 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:28:18.151025 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-bdced2bf-e0c7-4deb-861a-68c18db47848 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00819e68d8044dc98d2ce2264abe3f95'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:28:18.169817 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-bdced2bf-e0c7-4deb-861a-68c18db47848 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00819e68d8044dc98d2ce2264abe3f95'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:28:18.170725 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-bdced2bf-e0c7-4deb-861a-68c18db47848 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00819e68d8044dc98d2ce2264abe3f95'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:28:18.171327 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-bdced2bf-e0c7-4deb-861a-68c18db47848 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00819e68d8044dc98d2ce2264abe3f95'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:28:18.219934 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-bdced2bf-e0c7-4deb-861a-68c18db47848 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00819e68d8044dc98d2ce2264abe3f95'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:28:18.246407 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1120/2215] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:28:17 2022] DELETE /identity/v3/users/00819e68d8044dc98d2ce2264abe3f95 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:28:18.251515 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1913b9b-2b90-4511-aa15-3b0f0e3a9fde None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:18.252061 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1913b9b-2b90-4511-aa15-3b0f0e3a9fde None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:18.252200 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1913b9b-2b90-4511-aa15-3b0f0e3a9fde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:18.272074 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-f1913b9b-2b90-4511-aa15-3b0f0e3a9fde None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:28:18.273441 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-f1913b9b-2b90-4511-aa15-3b0f0e3a9fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:28:18.344953 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1096/2216] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:28:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:28:18.356236 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdced2bf-e0c7-4deb-861a-68c18db47848 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:28:18.357689 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-bdced2bf-e0c7-4deb-861a-68c18db47848 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:28:18.407980 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:28:18.426233 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdced2bf-e0c7-4deb-861a-68c18db47848 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:28:18.430356 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650b2610-405f-4f65-bdba-2603eac95682 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:28:18.430991 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650b2610-405f-4f65-bdba-2603eac95682 service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:18.431277 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650b2610-405f-4f65-bdba-2603eac95682 service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:18.431593 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650b2610-405f-4f65-bdba-2603eac95682 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:18.441529 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650b2610-405f-4f65-bdba-2603eac95682 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:28:18.442976 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650b2610-405f-4f65-bdba-2603eac95682 service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:28:18.459377 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1121/2217] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:28:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:28:18.976977 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1913b9b-2b90-4511-aa15-3b0f0e3a9fde None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:28:18.978720 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-f1913b9b-2b90-4511-aa15-3b0f0e3a9fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:28:19.019842 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1913b9b-2b90-4511-aa15-3b0f0e3a9fde None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:28:19.035174 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabffebe-ca7d-4fda-921c-127c67deb5d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:28:19.036105 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabffebe-ca7d-4fda-921c-127c67deb5d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:19.036393 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabffebe-ca7d-4fda-921c-127c67deb5d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:19.036694 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabffebe-ca7d-4fda-921c-127c67deb5d3 admin admin] PATH_INFO: `/v3/projects/371e9c7913f04a539c2ebd2c54b4672a` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:19.040210 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabffebe-ca7d-4fda-921c-127c67deb5d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=371e9c7913f04a539c2ebd2c54b4672a)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:28:19.042662 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabffebe-ca7d-4fda-921c-127c67deb5d3 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:28:19.168113 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-cabffebe-ca7d-4fda-921c-127c67deb5d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '371e9c7913f04a539c2ebd2c54b4672a', 'request_id': 'req-cabffebe-ca7d-4fda-921c-127c67deb5d3'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:28:19.182731 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-cabffebe-ca7d-4fda-921c-127c67deb5d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '371e9c7913f04a539c2ebd2c54b4672a', 'request_id': 'req-cabffebe-ca7d-4fda-921c-127c67deb5d3'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:28:19.277269 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-cabffebe-ca7d-4fda-921c-127c67deb5d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '371e9c7913f04a539c2ebd2c54b4672a', 'request_id': 'req-cabffebe-ca7d-4fda-921c-127c67deb5d3'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:28:19.292624 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-cabffebe-ca7d-4fda-921c-127c67deb5d3 admin admin] The token cache is being invalidate because project 371e9c7913f04a539c2ebd2c54b4672a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:28:19.292927 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-cabffebe-ca7d-4fda-921c-127c67deb5d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:28:19.293627 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-cabffebe-ca7d-4fda-921c-127c67deb5d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:28:19.295176 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1097/2218] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:28:18 2022] DELETE /identity/v3/projects/371e9c7913f04a539c2ebd2c54b4672a => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:28:19.300166 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233f23dd-7b8c-40e6-a86a-5f9715e1438b None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:19.301415 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233f23dd-7b8c-40e6-a86a-5f9715e1438b None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:19.301517 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233f23dd-7b8c-40e6-a86a-5f9715e1438b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:19.323607 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-233f23dd-7b8c-40e6-a86a-5f9715e1438b None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:28:19.324815 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-233f23dd-7b8c-40e6-a86a-5f9715e1438b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:28:19.395155 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1122/2219] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:28:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:28:19.404442 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35ca8c7-da8d-41b6-99b3-8794235b77a3 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:19.404730 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35ca8c7-da8d-41b6-99b3-8794235b77a3 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:19.405009 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35ca8c7-da8d-41b6-99b3-8794235b77a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:19.426682 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-c35ca8c7-da8d-41b6-99b3-8794235b77a3 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:28:19.427946 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-c35ca8c7-da8d-41b6-99b3-8794235b77a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:28:19.437270 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1098/2220] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:28:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:28:19.448615 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233f23dd-7b8c-40e6-a86a-5f9715e1438b None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:28:19.463023 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233f23dd-7b8c-40e6-a86a-5f9715e1438b None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:28:19.468358 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b9ce75-4b53-459b-9344-3b029fcbd42e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:28:19.468996 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b9ce75-4b53-459b-9344-3b029fcbd42e admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:19.469453 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b9ce75-4b53-459b-9344-3b029fcbd42e admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:19.469773 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b9ce75-4b53-459b-9344-3b029fcbd42e admin admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:19.470514 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b9ce75-4b53-459b-9344-3b029fcbd42e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:28:19.470822 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b9ce75-4b53-459b-9344-3b029fcbd42e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:28:19.473117 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b9ce75-4b53-459b-9344-3b029fcbd42e admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:28:19.485642 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1123/2221] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Aug 4 21:28:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:28:19.492990 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35ca8c7-da8d-41b6-99b3-8794235b77a3 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:28:19.507001 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35ca8c7-da8d-41b6-99b3-8794235b77a3 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:28:19.512834 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb98fe83-9049-4735-95e6-2b8df19fe0a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:28:19.513783 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb98fe83-9049-4735-95e6-2b8df19fe0a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:19.514208 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb98fe83-9049-4735-95e6-2b8df19fe0a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:19.514643 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb98fe83-9049-4735-95e6-2b8df19fe0a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:19.515882 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb98fe83-9049-4735-95e6-2b8df19fe0a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:28:19.518299 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb98fe83-9049-4735-95e6-2b8df19fe0a1 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:28:19.667834 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1099/2222] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:28:19 2022] POST /identity/v3/projects => generated 373 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:28:19.672716 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b9ce75-4b53-459b-9344-3b029fcbd42e admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:28:19.683922 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b9ce75-4b53-459b-9344-3b029fcbd42e admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:28:19.761709 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a69c66-4997-44a2-8798-007394523640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:28:19.762243 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a69c66-4997-44a2-8798-007394523640 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:19.762520 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a69c66-4997-44a2-8798-007394523640 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:19.762791 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a69c66-4997-44a2-8798-007394523640 admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:19.763497 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a69c66-4997-44a2-8798-007394523640 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:28:19.765460 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a69c66-4997-44a2-8798-007394523640 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:28:19.843285 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1124/2223] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:28:19 2022] POST /identity/v3/users => generated 287 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:28:19.848227 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb98fe83-9049-4735-95e6-2b8df19fe0a1 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:28:19.860888 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb98fe83-9049-4735-95e6-2b8df19fe0a1 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:28:19.868015 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c2df8f-0ff4-43e7-aac3-bab980698773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:28:19.868015 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c2df8f-0ff4-43e7-aac3-bab980698773 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:19.868015 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c2df8f-0ff4-43e7-aac3-bab980698773 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:19.868015 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c2df8f-0ff4-43e7-aac3-bab980698773 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:19.868934 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c2df8f-0ff4-43e7-aac3-bab980698773 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:28:19.870921 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c2df8f-0ff4-43e7-aac3-bab980698773 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:28:19.883142 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1100/2224] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:28:19 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:28:19.888777 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a69c66-4997-44a2-8798-007394523640 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:28:19.904202 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a69c66-4997-44a2-8798-007394523640 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:28:19.910842 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fe745d-9770-4336-ac2e-6478bf4b00f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:28:19.911789 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fe745d-9770-4336-ac2e-6478bf4b00f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:19.912138 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fe745d-9770-4336-ac2e-6478bf4b00f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:19.912477 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fe745d-9770-4336-ac2e-6478bf4b00f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:19.913294 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fe745d-9770-4336-ac2e-6478bf4b00f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:28:19.915399 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fe745d-9770-4336-ac2e-6478bf4b00f8 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:28:19.934858 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1125/2225] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:28:19 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:28:19.935271 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c2df8f-0ff4-43e7-aac3-bab980698773 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:28:19.947391 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c2df8f-0ff4-43e7-aac3-bab980698773 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:28:19.953880 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad25a024-8d3e-4b6b-bb5d-72518985cfd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:28:19.954647 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad25a024-8d3e-4b6b-bb5d-72518985cfd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:19.955002 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad25a024-8d3e-4b6b-bb5d-72518985cfd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:19.955280 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad25a024-8d3e-4b6b-bb5d-72518985cfd9 admin admin] PATH_INFO: `/v3/projects/1b1cf6e474724d94a8687e540bd49f7a/users/c1bee510a99844cc8e68da028ecb35cc/roles/8feebc60363c4feb870718b1721e5324` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:19.958135 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:28:19.982055 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad25a024-8d3e-4b6b-bb5d-72518985cfd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b1cf6e474724d94a8687e540bd49f7a, user_id=c1bee510a99844cc8e68da028ecb35cc, role_id=8feebc60363c4feb870718b1721e5324)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:28:19.987042 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad25a024-8d3e-4b6b-bb5d-72518985cfd9 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:28:20.071113 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1101/2226] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:28:19 2022] PUT /identity/v3/projects/1b1cf6e474724d94a8687e540bd49f7a/users/c1bee510a99844cc8e68da028ecb35cc/roles/8feebc60363c4feb870718b1721e5324 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:28:20.088052 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b9c503-076f-4d68-b03d-feb349a1ac7f None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:20.088052 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b9c503-076f-4d68-b03d-feb349a1ac7f None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:20.088052 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b9c503-076f-4d68-b03d-feb349a1ac7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:20.122572 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-a6b9c503-076f-4d68-b03d-feb349a1ac7f None None] MFA Rules not processed for user `c1bee510a99844cc8e68da028ecb35cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:28:20.123876 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-a6b9c503-076f-4d68-b03d-feb349a1ac7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:28:20.172209 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:28:20.215916 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1126/2227] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:28:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:28:20.228300 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad25a024-8d3e-4b6b-bb5d-72518985cfd9 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:28:20.230565 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-ad25a024-8d3e-4b6b-bb5d-72518985cfd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:28:20.335626 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad25a024-8d3e-4b6b-bb5d-72518985cfd9 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:28:20.339856 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca44ec8b-f491-4f84-9103-c40d128b26cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e31f2728056b460fa474478eb85ca1d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:28:20.340421 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca44ec8b-f491-4f84-9103-c40d128b26cf service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:20.340704 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca44ec8b-f491-4f84-9103-c40d128b26cf service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:20.340977 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca44ec8b-f491-4f84-9103-c40d128b26cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:20.357234 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca44ec8b-f491-4f84-9103-c40d128b26cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:28:20.358774 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca44ec8b-f491-4f84-9103-c40d128b26cf service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:28:20.369342 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1102/2228] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:28:20 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:28:20.417305 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b9c503-076f-4d68-b03d-feb349a1ac7f None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:28:20.491831 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b9c503-076f-4d68-b03d-feb349a1ac7f None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:28:20.498150 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbede6c1-6f84-4f6f-8b44-54c252eb1e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:28:20.498803 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbede6c1-6f84-4f6f-8b44-54c252eb1e8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:20.499090 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbede6c1-6f84-4f6f-8b44-54c252eb1e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:20.499511 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbede6c1-6f84-4f6f-8b44-54c252eb1e8c admin admin] PATH_INFO: `/v3/users/c1bee510a99844cc8e68da028ecb35cc` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:20.500901 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbede6c1-6f84-4f6f-8b44-54c252eb1e8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1bee510a99844cc8e68da028ecb35cc)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:28:20.503400 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbede6c1-6f84-4f6f-8b44-54c252eb1e8c admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:28:20.801066 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-cbede6c1-6f84-4f6f-8b44-54c252eb1e8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1bee510a99844cc8e68da028ecb35cc'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:28:20.813786 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-cbede6c1-6f84-4f6f-8b44-54c252eb1e8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1bee510a99844cc8e68da028ecb35cc'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:28:20.814539 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-cbede6c1-6f84-4f6f-8b44-54c252eb1e8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1bee510a99844cc8e68da028ecb35cc'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:28:20.815157 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-cbede6c1-6f84-4f6f-8b44-54c252eb1e8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1bee510a99844cc8e68da028ecb35cc'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:28:20.911499 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-cbede6c1-6f84-4f6f-8b44-54c252eb1e8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1bee510a99844cc8e68da028ecb35cc'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:28:20.929044 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1127/2229] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:28:20 2022] DELETE /identity/v3/users/c1bee510a99844cc8e68da028ecb35cc => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:28:20.933614 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa07ad3-7d65-4e37-a101-3676d2a82548 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:20.933901 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa07ad3-7d65-4e37-a101-3676d2a82548 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:20.934212 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa07ad3-7d65-4e37-a101-3676d2a82548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:20.960116 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-7aa07ad3-7d65-4e37-a101-3676d2a82548 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:28:20.961426 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-7aa07ad3-7d65-4e37-a101-3676d2a82548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:28:21.029129 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1103/2230] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:28:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:28:21.040277 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbede6c1-6f84-4f6f-8b44-54c252eb1e8c admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:28:21.041792 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-cbede6c1-6f84-4f6f-8b44-54c252eb1e8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:28:21.107187 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbede6c1-6f84-4f6f-8b44-54c252eb1e8c admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:28:21.111179 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce595f5-c0a5-46df-83d2-c0a47ec9c2fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:28:21.111941 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce595f5-c0a5-46df-83d2-c0a47ec9c2fd service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:21.112232 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce595f5-c0a5-46df-83d2-c0a47ec9c2fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:21.112512 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce595f5-c0a5-46df-83d2-c0a47ec9c2fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:21.130128 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce595f5-c0a5-46df-83d2-c0a47ec9c2fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:28:21.132050 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce595f5-c0a5-46df-83d2-c0a47ec9c2fd service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:28:21.136848 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:28:21.147692 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1128/2231] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:28:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:28:21.702360 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa07ad3-7d65-4e37-a101-3676d2a82548 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:28:21.703972 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-7aa07ad3-7d65-4e37-a101-3676d2a82548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:28:21.717819 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa07ad3-7d65-4e37-a101-3676d2a82548 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:28:21.730874 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0298b1-45e0-44b5-9c5b-9dcd91f9810c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:28:21.731571 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0298b1-45e0-44b5-9c5b-9dcd91f9810c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:21.731852 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0298b1-45e0-44b5-9c5b-9dcd91f9810c admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:21.732182 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0298b1-45e0-44b5-9c5b-9dcd91f9810c admin admin] PATH_INFO: `/v3/projects/1b1cf6e474724d94a8687e540bd49f7a` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:21.733366 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0298b1-45e0-44b5-9c5b-9dcd91f9810c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b1cf6e474724d94a8687e540bd49f7a)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:28:21.735286 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0298b1-45e0-44b5-9c5b-9dcd91f9810c admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:28:21.844064 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-db0298b1-45e0-44b5-9c5b-9dcd91f9810c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b1cf6e474724d94a8687e540bd49f7a', 'request_id': 'req-db0298b1-45e0-44b5-9c5b-9dcd91f9810c'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:28:21.850075 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-db0298b1-45e0-44b5-9c5b-9dcd91f9810c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b1cf6e474724d94a8687e540bd49f7a', 'request_id': 'req-db0298b1-45e0-44b5-9c5b-9dcd91f9810c'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:28:21.931419 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-db0298b1-45e0-44b5-9c5b-9dcd91f9810c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b1cf6e474724d94a8687e540bd49f7a', 'request_id': 'req-db0298b1-45e0-44b5-9c5b-9dcd91f9810c'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:28:21.943727 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-db0298b1-45e0-44b5-9c5b-9dcd91f9810c admin admin] The token cache is being invalidate because project 1b1cf6e474724d94a8687e540bd49f7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:28:21.944018 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-db0298b1-45e0-44b5-9c5b-9dcd91f9810c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:28:21.944754 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-db0298b1-45e0-44b5-9c5b-9dcd91f9810c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:28:21.946598 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1104/2232] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:28:21 2022] DELETE /identity/v3/projects/1b1cf6e474724d94a8687e540bd49f7a => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:28:21.951836 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d285a7a5-3065-4d2e-b5b0-686a09f73355 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:21.952360 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d285a7a5-3065-4d2e-b5b0-686a09f73355 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:21.952455 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d285a7a5-3065-4d2e-b5b0-686a09f73355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:21.973583 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-d285a7a5-3065-4d2e-b5b0-686a09f73355 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:28:21.974784 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-d285a7a5-3065-4d2e-b5b0-686a09f73355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:28:22.040121 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1129/2233] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:28:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:28:22.048294 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eefcf2-aea6-4757-81bd-9cacd8159181 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:22.048584 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eefcf2-aea6-4757-81bd-9cacd8159181 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:22.048868 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eefcf2-aea6-4757-81bd-9cacd8159181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:22.069763 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-46eefcf2-aea6-4757-81bd-9cacd8159181 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:28:22.070971 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-46eefcf2-aea6-4757-81bd-9cacd8159181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:28:22.079006 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1105/2234] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:28:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:28:22.084352 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d285a7a5-3065-4d2e-b5b0-686a09f73355 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:28:22.095037 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d285a7a5-3065-4d2e-b5b0-686a09f73355 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:28:22.101732 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a0cb20-727a-4a0e-bbe3-ad747ad6544e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:28:22.102368 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a0cb20-727a-4a0e-bbe3-ad747ad6544e admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:22.102652 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a0cb20-727a-4a0e-bbe3-ad747ad6544e admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:22.103044 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a0cb20-727a-4a0e-bbe3-ad747ad6544e admin admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:22.104050 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a0cb20-727a-4a0e-bbe3-ad747ad6544e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:28:22.104358 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a0cb20-727a-4a0e-bbe3-ad747ad6544e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:28:22.105987 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a0cb20-727a-4a0e-bbe3-ad747ad6544e admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:28:22.118219 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1130/2235] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Aug 4 21:28:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:28:22.122550 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eefcf2-aea6-4757-81bd-9cacd8159181 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:28:22.127116 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:28:22.132310 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eefcf2-aea6-4757-81bd-9cacd8159181 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:28:22.136905 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bcbbaf-5e02-41b4-adaa-825f2a9033ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:28:22.137510 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bcbbaf-5e02-41b4-adaa-825f2a9033ac admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:22.137788 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bcbbaf-5e02-41b4-adaa-825f2a9033ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:22.138100 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bcbbaf-5e02-41b4-adaa-825f2a9033ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:22.138781 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bcbbaf-5e02-41b4-adaa-825f2a9033ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:28:22.140709 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bcbbaf-5e02-41b4-adaa-825f2a9033ac admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:28:22.232190 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1106/2236] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:28:22 2022] POST /identity/v3/projects => generated 373 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:28:22.236313 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a0cb20-727a-4a0e-bbe3-ad747ad6544e admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:28:22.246983 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a0cb20-727a-4a0e-bbe3-ad747ad6544e admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:28:22.310347 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86686325-1b9a-4574-8577-b4c6754196f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:28:22.311099 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86686325-1b9a-4574-8577-b4c6754196f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:22.311436 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86686325-1b9a-4574-8577-b4c6754196f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:22.311720 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86686325-1b9a-4574-8577-b4c6754196f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:22.312409 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86686325-1b9a-4574-8577-b4c6754196f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:28:22.314376 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86686325-1b9a-4574-8577-b4c6754196f4 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:28:22.397475 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1131/2237] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:28:22 2022] POST /identity/v3/users => generated 287 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:28:22.402433 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bcbbaf-5e02-41b4-adaa-825f2a9033ac admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:28:22.414616 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bcbbaf-5e02-41b4-adaa-825f2a9033ac admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:28:22.420216 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f100e5-12ac-488b-bb95-49a53bb07633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:28:22.420874 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f100e5-12ac-488b-bb95-49a53bb07633 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:22.421161 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f100e5-12ac-488b-bb95-49a53bb07633 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:22.421440 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f100e5-12ac-488b-bb95-49a53bb07633 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:22.422144 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f100e5-12ac-488b-bb95-49a53bb07633 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:28:22.423856 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f100e5-12ac-488b-bb95-49a53bb07633 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:28:22.438456 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1107/2238] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:28:22 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:28:22.439784 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86686325-1b9a-4574-8577-b4c6754196f4 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:28:22.452127 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86686325-1b9a-4574-8577-b4c6754196f4 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:28:22.457929 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ce55cc-f117-464a-8ec8-9d80fe351779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:28:22.458564 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ce55cc-f117-464a-8ec8-9d80fe351779 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:22.458850 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ce55cc-f117-464a-8ec8-9d80fe351779 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:22.459132 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ce55cc-f117-464a-8ec8-9d80fe351779 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:22.459888 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ce55cc-f117-464a-8ec8-9d80fe351779 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:28:22.461514 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ce55cc-f117-464a-8ec8-9d80fe351779 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:28:22.474596 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1132/2239] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:28:22 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:28:22.478770 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f100e5-12ac-488b-bb95-49a53bb07633 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:28:22.489899 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f100e5-12ac-488b-bb95-49a53bb07633 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:28:22.495264 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf47bd3b-337d-4900-9305-44e4d4d9a66a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:28:22.496002 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf47bd3b-337d-4900-9305-44e4d4d9a66a admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:22.496284 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf47bd3b-337d-4900-9305-44e4d4d9a66a admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:22.496559 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf47bd3b-337d-4900-9305-44e4d4d9a66a admin admin] PATH_INFO: `/v3/projects/f1b4cfcf22614a95ad77db93922449cd/users/f480dbdc7237420680584ce4866b36e9/roles/8feebc60363c4feb870718b1721e5324` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:22.514380 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf47bd3b-337d-4900-9305-44e4d4d9a66a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1b4cfcf22614a95ad77db93922449cd, user_id=f480dbdc7237420680584ce4866b36e9, role_id=8feebc60363c4feb870718b1721e5324)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:28:22.517267 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf47bd3b-337d-4900-9305-44e4d4d9a66a admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:28:22.608829 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1108/2240] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:28:22 2022] PUT /identity/v3/projects/f1b4cfcf22614a95ad77db93922449cd/users/f480dbdc7237420680584ce4866b36e9/roles/8feebc60363c4feb870718b1721e5324 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:28:22.614494 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66c2272-1515-45b2-b542-e7a5d123c471 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:22.614782 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66c2272-1515-45b2-b542-e7a5d123c471 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:22.615049 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66c2272-1515-45b2-b542-e7a5d123c471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:22.635478 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-a66c2272-1515-45b2-b542-e7a5d123c471 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:28:22.636701 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-a66c2272-1515-45b2-b542-e7a5d123c471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:28:22.700024 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1133/2241] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:28:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:28:22.710755 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf47bd3b-337d-4900-9305-44e4d4d9a66a admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:28:22.712925 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-cf47bd3b-337d-4900-9305-44e4d4d9a66a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:28:22.774874 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf47bd3b-337d-4900-9305-44e4d4d9a66a admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:28:22.778476 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfea5095-5db4-4b6c-ae12-1ef32836dab6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:28:22.779242 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfea5095-5db4-4b6c-ae12-1ef32836dab6 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:22.779615 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfea5095-5db4-4b6c-ae12-1ef32836dab6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:22.779900 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfea5095-5db4-4b6c-ae12-1ef32836dab6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:22.789797 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfea5095-5db4-4b6c-ae12-1ef32836dab6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:28:22.791085 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfea5095-5db4-4b6c-ae12-1ef32836dab6 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:28:22.800832 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1109/2242] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:28:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:28:28.948956 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc6d490-4c04-43ef-8f1d-bf80beac6063 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:28.949504 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc6d490-4c04-43ef-8f1d-bf80beac6063 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:28.949504 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc6d490-4c04-43ef-8f1d-bf80beac6063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:28.957589 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:28:28.979597 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-acc6d490-4c04-43ef-8f1d-bf80beac6063 None None] MFA Rules not processed for user `f480dbdc7237420680584ce4866b36e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:28:28.980816 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-acc6d490-4c04-43ef-8f1d-bf80beac6063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:28:29.076309 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1134/2243] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:28:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:28:29.080871 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfea5095-5db4-4b6c-ae12-1ef32836dab6 service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:28:29.091835 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfea5095-5db4-4b6c-ae12-1ef32836dab6 service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:28:29.098153 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b08dde8-30b9-46fa-b2ab-394d305252a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:28:29.098796 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b08dde8-30b9-46fa-b2ab-394d305252a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:29.099083 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b08dde8-30b9-46fa-b2ab-394d305252a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:29.099395 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b08dde8-30b9-46fa-b2ab-394d305252a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:29.100085 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b08dde8-30b9-46fa-b2ab-394d305252a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:28:29.102095 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b08dde8-30b9-46fa-b2ab-394d305252a3 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:28:29.239051 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1110/2244] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:28:29 2022] POST /identity/v3/projects => generated 373 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:28:29.243086 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc6d490-4c04-43ef-8f1d-bf80beac6063 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:28:29.255260 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc6d490-4c04-43ef-8f1d-bf80beac6063 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:28:29.317579 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cb9f22-f0ff-457d-9277-cc5a9dd4b8f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:28:29.318289 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cb9f22-f0ff-457d-9277-cc5a9dd4b8f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:29.318645 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cb9f22-f0ff-457d-9277-cc5a9dd4b8f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:29.318930 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cb9f22-f0ff-457d-9277-cc5a9dd4b8f9 admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:29.319671 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cb9f22-f0ff-457d-9277-cc5a9dd4b8f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:28:29.321693 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cb9f22-f0ff-457d-9277-cc5a9dd4b8f9 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:28:29.422292 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1135/2245] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:28:29 2022] POST /identity/v3/users => generated 293 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:28:29.423540 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b08dde8-30b9-46fa-b2ab-394d305252a3 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:28:29.435510 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b08dde8-30b9-46fa-b2ab-394d305252a3 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:28:29.441401 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d0b302-3344-43a8-a2c8-a4115c4ec9d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:28:29.442057 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d0b302-3344-43a8-a2c8-a4115c4ec9d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:29.442345 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d0b302-3344-43a8-a2c8-a4115c4ec9d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:29.442620 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d0b302-3344-43a8-a2c8-a4115c4ec9d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:29.443386 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d0b302-3344-43a8-a2c8-a4115c4ec9d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:28:29.445187 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d0b302-3344-43a8-a2c8-a4115c4ec9d4 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:28:29.458167 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1111/2246] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:28:29 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:28:29.462589 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cb9f22-f0ff-457d-9277-cc5a9dd4b8f9 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:28:29.473108 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cb9f22-f0ff-457d-9277-cc5a9dd4b8f9 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:28:29.478065 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a6f0a0-c9fc-4044-8344-55e73c26dc82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:28:29.478836 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a6f0a0-c9fc-4044-8344-55e73c26dc82 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:29.479188 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a6f0a0-c9fc-4044-8344-55e73c26dc82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:29.479636 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a6f0a0-c9fc-4044-8344-55e73c26dc82 admin admin] PATH_INFO: `/v3/projects/ee3c311493db4aa987ec4ec5cc51547e/users/100ea03c7bfd4d41b799301f9ea5d655/roles/a432dd4735944ecfab2fa5ce1705d356` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:29.495623 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a6f0a0-c9fc-4044-8344-55e73c26dc82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee3c311493db4aa987ec4ec5cc51547e, user_id=100ea03c7bfd4d41b799301f9ea5d655, role_id=a432dd4735944ecfab2fa5ce1705d356)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:28:29.498360 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a6f0a0-c9fc-4044-8344-55e73c26dc82 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:28:29.574884 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1136/2247] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:28:29 2022] PUT /identity/v3/projects/ee3c311493db4aa987ec4ec5cc51547e/users/100ea03c7bfd4d41b799301f9ea5d655/roles/a432dd4735944ecfab2fa5ce1705d356 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:28:29.578858 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d0b302-3344-43a8-a2c8-a4115c4ec9d4 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:28:29.589503 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d0b302-3344-43a8-a2c8-a4115c4ec9d4 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:28:29.648258 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e096c0d-0bff-4f6f-9f5d-148fb811bed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:28:29.648954 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e096c0d-0bff-4f6f-9f5d-148fb811bed4 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:29.649480 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e096c0d-0bff-4f6f-9f5d-148fb811bed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:29.649761 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e096c0d-0bff-4f6f-9f5d-148fb811bed4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:29.650456 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e096c0d-0bff-4f6f-9f5d-148fb811bed4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:28:29.652112 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e096c0d-0bff-4f6f-9f5d-148fb811bed4 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:28:29.662112 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1112/2248] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:28:29 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:28:29.666280 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a6f0a0-c9fc-4044-8344-55e73c26dc82 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:28:29.677921 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a6f0a0-c9fc-4044-8344-55e73c26dc82 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:28:29.683270 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2664c9b4-3c9d-40c9-858e-50bb262c10cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:28:29.684011 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2664c9b4-3c9d-40c9-858e-50bb262c10cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:29.684299 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2664c9b4-3c9d-40c9-858e-50bb262c10cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:29.684577 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2664c9b4-3c9d-40c9-858e-50bb262c10cb admin admin] PATH_INFO: `/v3/projects/ee3c311493db4aa987ec4ec5cc51547e/users/100ea03c7bfd4d41b799301f9ea5d655/roles/a432dd4735944ecfab2fa5ce1705d356` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:29.686627 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2664c9b4-3c9d-40c9-858e-50bb262c10cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee3c311493db4aa987ec4ec5cc51547e, user_id=100ea03c7bfd4d41b799301f9ea5d655, role_id=a432dd4735944ecfab2fa5ce1705d356)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:28:29.689340 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2664c9b4-3c9d-40c9-858e-50bb262c10cb admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:28:29.703230 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1137/2249] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:28:29 2022] PUT /identity/v3/projects/ee3c311493db4aa987ec4ec5cc51547e/users/100ea03c7bfd4d41b799301f9ea5d655/roles/a432dd4735944ecfab2fa5ce1705d356 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:28:37.972016 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c134585a-ed5b-4146-81db-d57d5c1dffa9 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:37.973031 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c134585a-ed5b-4146-81db-d57d5c1dffa9 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:37.973474 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c134585a-ed5b-4146-81db-d57d5c1dffa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:37.998104 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:28:38.019738 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-c134585a-ed5b-4146-81db-d57d5c1dffa9 None None] MFA Rules not processed for user `100ea03c7bfd4d41b799301f9ea5d655`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:28:38.022630 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-c134585a-ed5b-4146-81db-d57d5c1dffa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:28:38.179626 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1113/2250] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:28:37 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:28:38.196196 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2664c9b4-3c9d-40c9-858e-50bb262c10cb admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:28:38.204015 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-2664c9b4-3c9d-40c9-858e-50bb262c10cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:28:38.279666 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:28:38.342385 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2664c9b4-3c9d-40c9-858e-50bb262c10cb admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:28:38.350196 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56019055-ae7c-4c99-a1d4-9986fd2305f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a2ce1df87fa47e5be301916bf3245fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:28:38.350984 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56019055-ae7c-4c99-a1d4-9986fd2305f3 service nova] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:38.351392 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56019055-ae7c-4c99-a1d4-9986fd2305f3 service nova] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:38.351677 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56019055-ae7c-4c99-a1d4-9986fd2305f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:38.417506 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56019055-ae7c-4c99-a1d4-9986fd2305f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:28:38.423463 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56019055-ae7c-4c99-a1d4-9986fd2305f3 service nova] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:28:38.433709 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1138/2251] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:28:38 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:28:41.933053 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b3cc8-5883-4bc2-bde3-4321c4b9cba5 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:41.933514 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b3cc8-5883-4bc2-bde3-4321c4b9cba5 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:41.933607 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b3cc8-5883-4bc2-bde3-4321c4b9cba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:41.960895 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-536b3cc8-5883-4bc2-bde3-4321c4b9cba5 None None] MFA Rules not processed for user `23ea380385f649628cc27e3603bc700c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:28:41.960895 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-536b3cc8-5883-4bc2-bde3-4321c4b9cba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:28:41.997812 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2033f5-ed30-4609-9149-2a254d78b640 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:41.998365 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2033f5-ed30-4609-9149-2a254d78b640 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:41.998801 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2033f5-ed30-4609-9149-2a254d78b640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:42.038223 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-3a2033f5-ed30-4609-9149-2a254d78b640 None None] MFA Rules not processed for user `f480dbdc7237420680584ce4866b36e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:28:42.040432 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-3a2033f5-ed30-4609-9149-2a254d78b640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:28:42.055763 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1139/2252] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:28:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:28:42.073431 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1114/2253] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Aug 4 21:28:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:28:42.089344 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2033f5-ed30-4609-9149-2a254d78b640 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:28:42.089637 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-3a2033f5-ed30-4609-9149-2a254d78b640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:28:42.241372 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2033f5-ed30-4609-9149-2a254d78b640 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:28:42.247792 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73222763-44f4-421d-926e-a1d40cb40d32 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58935367c0ad445f8477cdac0df04d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:28:42.248520 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73222763-44f4-421d-926e-a1d40cb40d32 service swift] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:42.248884 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73222763-44f4-421d-926e-a1d40cb40d32 service swift] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:42.249241 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73222763-44f4-421d-926e-a1d40cb40d32 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:42.261549 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73222763-44f4-421d-926e-a1d40cb40d32 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:28:42.263898 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73222763-44f4-421d-926e-a1d40cb40d32 service swift] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:28:42.273712 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1140/2254] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Aug 4 21:28:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 21:28:43.720425 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e2dbff-a905-466c-b2ed-edd84c81e9b9 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:43.721007 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e2dbff-a905-466c-b2ed-edd84c81e9b9 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:43.721337 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e2dbff-a905-466c-b2ed-edd84c81e9b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:43.754606 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-81e2dbff-a905-466c-b2ed-edd84c81e9b9 None None] MFA Rules not processed for user `23ea380385f649628cc27e3603bc700c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:28:43.756228 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-81e2dbff-a905-466c-b2ed-edd84c81e9b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:28:43.766947 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1115/2255] 10.222.0.25 () {58 vars in 967 bytes} [Thu Aug 4 21:28:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:28:43.789641 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73222763-44f4-421d-926e-a1d40cb40d32 service swift] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:28:43.805914 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73222763-44f4-421d-926e-a1d40cb40d32 service swift] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:28:43.811554 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2631676c-6eaa-4b4c-a799-fc278190539b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58935367c0ad445f8477cdac0df04d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:28:43.812468 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2631676c-6eaa-4b4c-a799-fc278190539b service swift] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:43.813008 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2631676c-6eaa-4b4c-a799-fc278190539b service swift] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:43.813956 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2631676c-6eaa-4b4c-a799-fc278190539b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:43.824185 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2631676c-6eaa-4b4c-a799-fc278190539b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:28:43.825884 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2631676c-6eaa-4b4c-a799-fc278190539b service swift] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:28:43.837286 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1141/2256] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Aug 4 21:28:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 21:28:53.747249 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=57 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:28:53.833362 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=76 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:28:56.226392 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c445397c-abbc-43cd-83c7-a5d9a4935d41 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:28:56.227411 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c445397c-abbc-43cd-83c7-a5d9a4935d41 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:28:56.229511 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c445397c-abbc-43cd-83c7-a5d9a4935d41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:28:56.386581 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-c445397c-abbc-43cd-83c7-a5d9a4935d41 None None] MFA Rules not processed for user `ea2f8480bc7d44f6a51e4a1d875a1c7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:28:56.388129 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-c445397c-abbc-43cd-83c7-a5d9a4935d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:28:56.462920 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1116/2257] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:28:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:29:06.454714 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=25 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:29:06.905259 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8661e2f6-3ebf-45f3-a5bb-ff61264dc971 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:29:06.906144 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8661e2f6-3ebf-45f3-a5bb-ff61264dc971 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:29:06.906619 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8661e2f6-3ebf-45f3-a5bb-ff61264dc971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:29:06.927454 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-8661e2f6-3ebf-45f3-a5bb-ff61264dc971 None None] MFA Rules not processed for user `23ea380385f649628cc27e3603bc700c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:29:06.928705 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-8661e2f6-3ebf-45f3-a5bb-ff61264dc971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:29:06.937201 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1142/2258] 10.222.0.25 () {58 vars in 967 bytes} [Thu Aug 4 21:29:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:29:06.950953 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c445397c-abbc-43cd-83c7-a5d9a4935d41 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:29:06.964109 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c445397c-abbc-43cd-83c7-a5d9a4935d41 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:29:06.969851 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19268c08-6024-4d0d-a9ca-0901acd237b2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58935367c0ad445f8477cdac0df04d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:29:06.970482 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19268c08-6024-4d0d-a9ca-0901acd237b2 service swift] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:29:06.970768 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19268c08-6024-4d0d-a9ca-0901acd237b2 service swift] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:29:06.971257 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19268c08-6024-4d0d-a9ca-0901acd237b2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:29:06.982238 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19268c08-6024-4d0d-a9ca-0901acd237b2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:29:06.985900 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19268c08-6024-4d0d-a9ca-0901acd237b2 service swift] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:29:06.999604 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1117/2259] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Aug 4 21:29:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 21:29:10.447308 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8661e2f6-3ebf-45f3-a5bb-ff61264dc971 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:29:10.451096 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-8661e2f6-3ebf-45f3-a5bb-ff61264dc971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:29:10.533304 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8661e2f6-3ebf-45f3-a5bb-ff61264dc971 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:29:10.540412 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c4eabc-3331-4f4b-afeb-84296f53fcfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:29:10.541293 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c4eabc-3331-4f4b-afeb-84296f53fcfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:29:10.541674 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c4eabc-3331-4f4b-afeb-84296f53fcfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:29:10.541965 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c4eabc-3331-4f4b-afeb-84296f53fcfe admin admin] PATH_INFO: `/v3/users/1039d83bfa58450ebe10e727f1dc0ea5` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:29:10.543189 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c4eabc-3331-4f4b-afeb-84296f53fcfe admin admin] RBAC: Authorizing `identity:delete_user(user_id=1039d83bfa58450ebe10e727f1dc0ea5)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:29:10.545312 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c4eabc-3331-4f4b-afeb-84296f53fcfe admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:29:10.724230 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-05c4eabc-3331-4f4b-afeb-84296f53fcfe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1039d83bfa58450ebe10e727f1dc0ea5'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:29:10.737096 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-05c4eabc-3331-4f4b-afeb-84296f53fcfe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1039d83bfa58450ebe10e727f1dc0ea5'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:29:10.738464 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-05c4eabc-3331-4f4b-afeb-84296f53fcfe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1039d83bfa58450ebe10e727f1dc0ea5'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:29:10.739162 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-05c4eabc-3331-4f4b-afeb-84296f53fcfe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1039d83bfa58450ebe10e727f1dc0ea5'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:29:10.779861 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-05c4eabc-3331-4f4b-afeb-84296f53fcfe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1039d83bfa58450ebe10e727f1dc0ea5'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:29:10.794568 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1143/2260] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:29:10 2022] DELETE /identity/v3/users/1039d83bfa58450ebe10e727f1dc0ea5 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:29:11.103036 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19268c08-6024-4d0d-a9ca-0901acd237b2 service swift] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:29:11.132118 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19268c08-6024-4d0d-a9ca-0901acd237b2 service swift] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:29:11.216583 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a419a9-51c0-4b48-a049-79f3f7dcff39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:29:11.217402 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a419a9-51c0-4b48-a049-79f3f7dcff39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:29:11.217815 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a419a9-51c0-4b48-a049-79f3f7dcff39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:29:11.218184 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a419a9-51c0-4b48-a049-79f3f7dcff39 admin admin] PATH_INFO: `/v3/projects/e734c237455f4d42b1e34eeb6ed38511` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:29:11.224296 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a419a9-51c0-4b48-a049-79f3f7dcff39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e734c237455f4d42b1e34eeb6ed38511)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:29:11.230799 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a419a9-51c0-4b48-a049-79f3f7dcff39 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:29:11.356287 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-e8a419a9-51c0-4b48-a049-79f3f7dcff39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e734c237455f4d42b1e34eeb6ed38511', 'request_id': 'req-e8a419a9-51c0-4b48-a049-79f3f7dcff39'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:29:11.361874 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-e8a419a9-51c0-4b48-a049-79f3f7dcff39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e734c237455f4d42b1e34eeb6ed38511', 'request_id': 'req-e8a419a9-51c0-4b48-a049-79f3f7dcff39'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:29:11.446971 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-e8a419a9-51c0-4b48-a049-79f3f7dcff39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e734c237455f4d42b1e34eeb6ed38511', 'request_id': 'req-e8a419a9-51c0-4b48-a049-79f3f7dcff39'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:29:11.462560 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-e8a419a9-51c0-4b48-a049-79f3f7dcff39 admin admin] The token cache is being invalidate because project e734c237455f4d42b1e34eeb6ed38511 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:29:11.462959 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-e8a419a9-51c0-4b48-a049-79f3f7dcff39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:29:11.463736 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-e8a419a9-51c0-4b48-a049-79f3f7dcff39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:29:11.465412 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1118/2261] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:29:11 2022] DELETE /identity/v3/projects/e734c237455f4d42b1e34eeb6ed38511 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:29:20.790324 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=56,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:29:21.458785 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:29:28.691708 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c4eabc-3331-4f4b-afeb-84296f53fcfe admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:29:28.693539 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-05c4eabc-3331-4f4b-afeb-84296f53fcfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:29:28.751786 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c4eabc-3331-4f4b-afeb-84296f53fcfe admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:29:28.804220 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a963a7-50af-4bd8-bd65-e9d69dcfccab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:29:28.804848 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a963a7-50af-4bd8-bd65-e9d69dcfccab service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:29:28.805130 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a963a7-50af-4bd8-bd65-e9d69dcfccab service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:29:28.805403 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a963a7-50af-4bd8-bd65-e9d69dcfccab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:29:28.807179 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-08a963a7-50af-4bd8-bd65-e9d69dcfccab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:29:28.873289 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a963a7-50af-4bd8-bd65-e9d69dcfccab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:29:28.874744 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a963a7-50af-4bd8-bd65-e9d69dcfccab service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:29:28.891260 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1144/2262] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:29:28 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:29:37.492021 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a85e271-bd34-40df-a2ac-43e1e8c1d819 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:29:37.492544 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a85e271-bd34-40df-a2ac-43e1e8c1d819 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:29:37.492626 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a85e271-bd34-40df-a2ac-43e1e8c1d819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:29:37.512021 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-8a85e271-bd34-40df-a2ac-43e1e8c1d819 None None] MFA Rules not processed for user `3d5f53509b2f42269cd26e77c79ad8be`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:29:37.513201 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-8a85e271-bd34-40df-a2ac-43e1e8c1d819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:29:37.574431 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1119/2263] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:29:37 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:29:38.885382 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=70 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:29:40.868092 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15c03a6-f158-4b98-8f0f-6a5d564b46db None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:29:40.868714 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15c03a6-f158-4b98-8f0f-6a5d564b46db None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:29:40.868714 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15c03a6-f158-4b98-8f0f-6a5d564b46db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:29:40.890389 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-a15c03a6-f158-4b98-8f0f-6a5d564b46db None None] MFA Rules not processed for user `4087b29e666c4afaa36118e82eb169d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:29:40.891657 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-a15c03a6-f158-4b98-8f0f-6a5d564b46db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:29:40.950567 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1145/2264] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:29:40 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:29:47.563149 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=25 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:29:50.942857 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=25 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:30:04.124921 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a85e271-bd34-40df-a2ac-43e1e8c1d819 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:30:04.126958 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-8a85e271-bd34-40df-a2ac-43e1e8c1d819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:30:04.222255 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a85e271-bd34-40df-a2ac-43e1e8c1d819 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:30:04.227219 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c782615-eef3-4665-95ba-2cfbf6ebb184 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e31f2728056b460fa474478eb85ca1d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:30:04.228107 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c782615-eef3-4665-95ba-2cfbf6ebb184 service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:30:04.228381 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c782615-eef3-4665-95ba-2cfbf6ebb184 service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:30:04.228654 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c782615-eef3-4665-95ba-2cfbf6ebb184 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:30:04.231005 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-9c782615-eef3-4665-95ba-2cfbf6ebb184 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:30:04.318615 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c782615-eef3-4665-95ba-2cfbf6ebb184 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:30:04.320805 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c782615-eef3-4665-95ba-2cfbf6ebb184 service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:30:04.336937 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1120/2265] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:30:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:30:09.010757 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a328d944-6e14-41e4-a487-2edd4a9521df None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:30:09.011309 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a328d944-6e14-41e4-a487-2edd4a9521df None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:30:09.011309 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a328d944-6e14-41e4-a487-2edd4a9521df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:30:09.030308 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-a328d944-6e14-41e4-a487-2edd4a9521df None None] MFA Rules not processed for user `9b29c3f2c29d4e2aba552c144cff94f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:30:09.031551 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-a328d944-6e14-41e4-a487-2edd4a9521df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:30:09.090433 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1146/2266] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:30:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:30:09.802792 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c782615-eef3-4665-95ba-2cfbf6ebb184 service cinder] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:30:09.804302 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-9c782615-eef3-4665-95ba-2cfbf6ebb184 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:30:09.815977 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c782615-eef3-4665-95ba-2cfbf6ebb184 service cinder] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:30:09.820009 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1a81e2-eba3-4fb4-9f98-df1c17c06590 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:30:09.820736 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1a81e2-eba3-4fb4-9f98-df1c17c06590 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:30:09.821116 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1a81e2-eba3-4fb4-9f98-df1c17c06590 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:30:09.821462 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1a81e2-eba3-4fb4-9f98-df1c17c06590 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:30:09.823448 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-9c1a81e2-eba3-4fb4-9f98-df1c17c06590 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:30:09.839013 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1a81e2-eba3-4fb4-9f98-df1c17c06590 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:30:09.840658 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1a81e2-eba3-4fb4-9f98-df1c17c06590 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:30:09.853377 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1121/2267] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:30:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:30:19.083459 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=25 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:30:19.174982 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a328d944-6e14-41e4-a487-2edd4a9521df None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:30:19.179711 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-a328d944-6e14-41e4-a487-2edd4a9521df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:30:19.210505 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a328d944-6e14-41e4-a487-2edd4a9521df None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:30:19.221767 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60276d24-b6f8-4b73-af23-ffc134132e92 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a2ce1df87fa47e5be301916bf3245fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:30:19.221767 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60276d24-b6f8-4b73-af23-ffc134132e92 service nova] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:30:19.221767 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60276d24-b6f8-4b73-af23-ffc134132e92 service nova] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:30:19.221767 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60276d24-b6f8-4b73-af23-ffc134132e92 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:30:19.222256 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-60276d24-b6f8-4b73-af23-ffc134132e92 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:30:19.430787 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60276d24-b6f8-4b73-af23-ffc134132e92 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:30:19.430787 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60276d24-b6f8-4b73-af23-ffc134132e92 service nova] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:30:19.483154 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1147/2268] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:30:19 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:30:19.847688 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=60 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:30:23.948635 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1a81e2-eba3-4fb4-9f98-df1c17c06590 service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:30:23.959590 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1a81e2-eba3-4fb4-9f98-df1c17c06590 service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:30:23.963416 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df13ab4b-0ba2-403f-90f9-2a7140cf8e2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e31f2728056b460fa474478eb85ca1d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:30:23.968505 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df13ab4b-0ba2-403f-90f9-2a7140cf8e2f service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:30:23.968505 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df13ab4b-0ba2-403f-90f9-2a7140cf8e2f service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:30:23.968505 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df13ab4b-0ba2-403f-90f9-2a7140cf8e2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:30:23.968505 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-df13ab4b-0ba2-403f-90f9-2a7140cf8e2f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:30:23.971072 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2022ffc-384f-4ac2-9670-5c8fb4e9cdab None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:30:23.971588 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2022ffc-384f-4ac2-9670-5c8fb4e9cdab None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:30:23.971864 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2022ffc-384f-4ac2-9670-5c8fb4e9cdab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:30:23.978870 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df13ab4b-0ba2-403f-90f9-2a7140cf8e2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:30:23.980646 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df13ab4b-0ba2-403f-90f9-2a7140cf8e2f service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:30:23.991798 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1122/2269] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:30:23 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:30:23.993903 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-d2022ffc-384f-4ac2-9670-5c8fb4e9cdab None None] MFA Rules not processed for user `ea2f8480bc7d44f6a51e4a1d875a1c7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:30:23.995165 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-d2022ffc-384f-4ac2-9670-5c8fb4e9cdab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:30:24.003144 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1148/2270] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:30:23 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:30:24.328295 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df13ab4b-0ba2-403f-90f9-2a7140cf8e2f service cinder] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:30:24.329701 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-df13ab4b-0ba2-403f-90f9-2a7140cf8e2f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:30:24.342382 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df13ab4b-0ba2-403f-90f9-2a7140cf8e2f service cinder] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:30:24.348677 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fabbc63-fc75-4132-919b-fc395eff3c8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:30:24.348677 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fabbc63-fc75-4132-919b-fc395eff3c8b service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:30:24.348677 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fabbc63-fc75-4132-919b-fc395eff3c8b service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:30:24.348677 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fabbc63-fc75-4132-919b-fc395eff3c8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:30:24.349063 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-9fabbc63-fc75-4132-919b-fc395eff3c8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:30:24.358228 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fabbc63-fc75-4132-919b-fc395eff3c8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:30:24.359673 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fabbc63-fc75-4132-919b-fc395eff3c8b service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:30:24.368861 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1123/2271] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:30:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:30:24.813384 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2022ffc-384f-4ac2-9670-5c8fb4e9cdab None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:30:24.814910 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-d2022ffc-384f-4ac2-9670-5c8fb4e9cdab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:30:25.025175 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2022ffc-384f-4ac2-9670-5c8fb4e9cdab None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:30:25.030616 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c84679d-9a50-4983-8848-866e29d5239b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67d44a0098ab4c36b14b36e9e59ae75c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:30:25.032017 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c84679d-9a50-4983-8848-866e29d5239b service placement] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:30:25.032017 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c84679d-9a50-4983-8848-866e29d5239b service placement] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:30:25.032017 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c84679d-9a50-4983-8848-866e29d5239b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:30:25.034410 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-2c84679d-9a50-4983-8848-866e29d5239b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:30:25.048323 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c84679d-9a50-4983-8848-866e29d5239b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:30:25.051247 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c84679d-9a50-4983-8848-866e29d5239b service placement] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:30:25.062074 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1149/2272] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:30:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:30:29.226224 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fabbc63-fc75-4132-919b-fc395eff3c8b service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:30:29.237755 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fabbc63-fc75-4132-919b-fc395eff3c8b service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:30:29.241173 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a2bd8e-5cfb-433a-9ad1-6e89dafbec86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:30:29.241758 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a2bd8e-5cfb-433a-9ad1-6e89dafbec86 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:30:29.242092 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a2bd8e-5cfb-433a-9ad1-6e89dafbec86 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:30:29.242294 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a2bd8e-5cfb-433a-9ad1-6e89dafbec86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:30:29.244097 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-33a2bd8e-5cfb-433a-9ad1-6e89dafbec86 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:30:29.254676 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a2bd8e-5cfb-433a-9ad1-6e89dafbec86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:30:29.256211 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a2bd8e-5cfb-433a-9ad1-6e89dafbec86 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:30:29.265828 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1124/2273] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:30:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:30:35.057539 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=63 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:30:38.544539 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c84679d-9a50-4983-8848-866e29d5239b service placement] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:30:38.546099 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-2c84679d-9a50-4983-8848-866e29d5239b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:30:38.563292 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c84679d-9a50-4983-8848-866e29d5239b service placement] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:30:38.571160 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5c7497-4be6-4d0e-89bd-80f02e8baa05 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a2ce1df87fa47e5be301916bf3245fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:30:38.571914 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5c7497-4be6-4d0e-89bd-80f02e8baa05 service nova] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:30:38.572820 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5c7497-4be6-4d0e-89bd-80f02e8baa05 service nova] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:30:38.572820 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5c7497-4be6-4d0e-89bd-80f02e8baa05 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:30:38.574839 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-ac5c7497-4be6-4d0e-89bd-80f02e8baa05 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:30:38.587723 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5c7497-4be6-4d0e-89bd-80f02e8baa05 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:30:38.589472 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5c7497-4be6-4d0e-89bd-80f02e8baa05 service nova] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:30:38.603009 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1150/2274] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:30:38 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:30:39.261242 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=18 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:30:39.384112 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a2bd8e-5cfb-433a-9ad1-6e89dafbec86 service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:30:39.401702 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a2bd8e-5cfb-433a-9ad1-6e89dafbec86 service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:30:39.410779 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a65ef95-d421-404f-a84f-ebc937f8cd14 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a2ce1df87fa47e5be301916bf3245fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:30:39.411722 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a65ef95-d421-404f-a84f-ebc937f8cd14 service nova] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:30:39.412186 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a65ef95-d421-404f-a84f-ebc937f8cd14 service nova] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:30:39.412630 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a65ef95-d421-404f-a84f-ebc937f8cd14 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:30:39.414928 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-2a65ef95-d421-404f-a84f-ebc937f8cd14 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:30:39.434554 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a65ef95-d421-404f-a84f-ebc937f8cd14 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:30:39.437343 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a65ef95-d421-404f-a84f-ebc937f8cd14 service nova] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:30:39.451208 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1125/2275] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:30:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:30:42.839210 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5c7497-4be6-4d0e-89bd-80f02e8baa05 service nova] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:30:42.852686 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5c7497-4be6-4d0e-89bd-80f02e8baa05 service nova] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:30:42.859188 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c02694c-c93b-452e-9023-5dab81a20081 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a2ce1df87fa47e5be301916bf3245fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:30:42.860044 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c02694c-c93b-452e-9023-5dab81a20081 service nova] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:30:42.860353 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c02694c-c93b-452e-9023-5dab81a20081 service nova] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:30:42.860815 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c02694c-c93b-452e-9023-5dab81a20081 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:30:42.862620 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-6c02694c-c93b-452e-9023-5dab81a20081 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:30:42.885652 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c02694c-c93b-452e-9023-5dab81a20081 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:30:42.888446 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c02694c-c93b-452e-9023-5dab81a20081 service nova] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:30:42.908415 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1151/2276] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:30:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:30:48.433539 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a65ef95-d421-404f-a84f-ebc937f8cd14 service nova] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:30:48.443529 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a65ef95-d421-404f-a84f-ebc937f8cd14 service nova] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:30:48.447407 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a05ba99-3653-4dcb-9f9e-74e62aeaf2e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:30:48.447966 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a05ba99-3653-4dcb-9f9e-74e62aeaf2e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:30:48.448245 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a05ba99-3653-4dcb-9f9e-74e62aeaf2e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:30:48.448519 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a05ba99-3653-4dcb-9f9e-74e62aeaf2e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:30:48.450244 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-0a05ba99-3653-4dcb-9f9e-74e62aeaf2e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:30:48.514939 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a05ba99-3653-4dcb-9f9e-74e62aeaf2e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:30:48.516431 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a05ba99-3653-4dcb-9f9e-74e62aeaf2e7 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:30:48.526004 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1126/2277] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:30:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:30:52.323954 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c02694c-c93b-452e-9023-5dab81a20081 service nova] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:30:52.328163 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-6c02694c-c93b-452e-9023-5dab81a20081 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:30:52.342642 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c02694c-c93b-452e-9023-5dab81a20081 service nova] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:30:52.347908 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbcbcf9-1680-4634-9c29-840c26497641 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67d44a0098ab4c36b14b36e9e59ae75c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:30:52.348590 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbcbcf9-1680-4634-9c29-840c26497641 service placement] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:30:52.348959 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbcbcf9-1680-4634-9c29-840c26497641 service placement] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:30:52.349150 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbcbcf9-1680-4634-9c29-840c26497641 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:30:52.350964 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-cbbcbcf9-1680-4634-9c29-840c26497641 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:30:52.366302 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbcbcf9-1680-4634-9c29-840c26497641 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:30:52.367944 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbcbcf9-1680-4634-9c29-840c26497641 service placement] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:30:52.379250 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1152/2278] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:30:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:30:52.457615 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a05ba99-3653-4dcb-9f9e-74e62aeaf2e7 service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:30:52.459565 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-0a05ba99-3653-4dcb-9f9e-74e62aeaf2e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:30:52.473516 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a05ba99-3653-4dcb-9f9e-74e62aeaf2e7 service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:30:52.478756 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205c9d6f-0d6f-46b3-87db-c847c5b2ed16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:30:52.479110 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205c9d6f-0d6f-46b3-87db-c847c5b2ed16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:30:52.479427 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205c9d6f-0d6f-46b3-87db-c847c5b2ed16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:30:52.479707 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205c9d6f-0d6f-46b3-87db-c847c5b2ed16 admin admin] PATH_INFO: `/v3/users/3d5f53509b2f42269cd26e77c79ad8be` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:30:52.480891 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205c9d6f-0d6f-46b3-87db-c847c5b2ed16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d5f53509b2f42269cd26e77c79ad8be)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:30:52.482819 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205c9d6f-0d6f-46b3-87db-c847c5b2ed16 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:30:52.675772 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-205c9d6f-0d6f-46b3-87db-c847c5b2ed16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d5f53509b2f42269cd26e77c79ad8be'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:30:52.693576 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-205c9d6f-0d6f-46b3-87db-c847c5b2ed16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d5f53509b2f42269cd26e77c79ad8be'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:30:52.694585 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-205c9d6f-0d6f-46b3-87db-c847c5b2ed16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d5f53509b2f42269cd26e77c79ad8be'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:30:52.705205 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-205c9d6f-0d6f-46b3-87db-c847c5b2ed16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d5f53509b2f42269cd26e77c79ad8be'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:30:52.705929 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-205c9d6f-0d6f-46b3-87db-c847c5b2ed16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d5f53509b2f42269cd26e77c79ad8be'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:30:52.803251 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1127/2279] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:30:52 2022] DELETE /identity/v3/users/3d5f53509b2f42269cd26e77c79ad8be => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:30:53.044100 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbcbcf9-1680-4634-9c29-840c26497641 service placement] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:30:53.057025 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbcbcf9-1680-4634-9c29-840c26497641 service placement] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:30:53.118191 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cf1963-a5b5-4117-a189-12c4e43804e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:30:53.118838 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cf1963-a5b5-4117-a189-12c4e43804e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:30:53.119123 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cf1963-a5b5-4117-a189-12c4e43804e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:30:53.119427 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cf1963-a5b5-4117-a189-12c4e43804e7 admin admin] PATH_INFO: `/v3/projects/0cc9fe2e0830451c823180afe60ec921` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:30:53.120588 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cf1963-a5b5-4117-a189-12c4e43804e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0cc9fe2e0830451c823180afe60ec921)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:30:53.122481 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cf1963-a5b5-4117-a189-12c4e43804e7 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:30:53.245958 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-a3cf1963-a5b5-4117-a189-12c4e43804e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0cc9fe2e0830451c823180afe60ec921', 'request_id': 'req-a3cf1963-a5b5-4117-a189-12c4e43804e7'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:30:53.252027 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-a3cf1963-a5b5-4117-a189-12c4e43804e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0cc9fe2e0830451c823180afe60ec921', 'request_id': 'req-a3cf1963-a5b5-4117-a189-12c4e43804e7'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:30:53.345590 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-a3cf1963-a5b5-4117-a189-12c4e43804e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0cc9fe2e0830451c823180afe60ec921', 'request_id': 'req-a3cf1963-a5b5-4117-a189-12c4e43804e7'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:30:53.358989 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-a3cf1963-a5b5-4117-a189-12c4e43804e7 admin admin] The token cache is being invalidate because project 0cc9fe2e0830451c823180afe60ec921 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:30:53.359279 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-a3cf1963-a5b5-4117-a189-12c4e43804e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:30:53.359983 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-a3cf1963-a5b5-4117-a189-12c4e43804e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:30:53.361522 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1153/2280] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:30:53 2022] DELETE /identity/v3/projects/0cc9fe2e0830451c823180afe60ec921 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:30:53.366468 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0928553-fff2-4920-a24b-918a35bcf4a8 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:30:53.367029 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0928553-fff2-4920-a24b-918a35bcf4a8 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:30:53.367029 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0928553-fff2-4920-a24b-918a35bcf4a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:30:53.395779 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-c0928553-fff2-4920-a24b-918a35bcf4a8 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:30:53.396981 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-c0928553-fff2-4920-a24b-918a35bcf4a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:30:53.443145 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:30:53.457361 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1128/2281] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:30:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:30:53.465265 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d25e7a-3dfd-453f-82d8-3d69dacd6d7d None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:30:53.465550 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d25e7a-3dfd-453f-82d8-3d69dacd6d7d None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:30:53.465822 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d25e7a-3dfd-453f-82d8-3d69dacd6d7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:30:53.485235 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-59d25e7a-3dfd-453f-82d8-3d69dacd6d7d None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:30:53.486425 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-59d25e7a-3dfd-453f-82d8-3d69dacd6d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:30:53.494169 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1154/2282] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:30:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:30:53.498863 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0928553-fff2-4920-a24b-918a35bcf4a8 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:30:53.509408 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0928553-fff2-4920-a24b-918a35bcf4a8 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:30:53.514506 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c6b4de-9e72-4089-86df-ca372f9b4392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:30:53.515144 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c6b4de-9e72-4089-86df-ca372f9b4392 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:30:53.515464 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c6b4de-9e72-4089-86df-ca372f9b4392 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:30:53.515768 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c6b4de-9e72-4089-86df-ca372f9b4392 admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:30:53.516458 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c6b4de-9e72-4089-86df-ca372f9b4392 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:30:53.516766 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c6b4de-9e72-4089-86df-ca372f9b4392 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:30:53.519009 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c6b4de-9e72-4089-86df-ca372f9b4392 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:30:53.530776 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1129/2283] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Aug 4 21:30:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:30:53.535090 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d25e7a-3dfd-453f-82d8-3d69dacd6d7d None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:30:53.544960 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d25e7a-3dfd-453f-82d8-3d69dacd6d7d None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:30:53.550004 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491f317c-cdfc-41f1-bc86-4dfc15d338a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:30:53.551910 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491f317c-cdfc-41f1-bc86-4dfc15d338a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:30:53.551910 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491f317c-cdfc-41f1-bc86-4dfc15d338a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:30:53.551910 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491f317c-cdfc-41f1-bc86-4dfc15d338a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:30:53.551910 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491f317c-cdfc-41f1-bc86-4dfc15d338a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:30:53.554444 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491f317c-cdfc-41f1-bc86-4dfc15d338a6 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:30:53.673847 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1155/2284] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:30:53 2022] POST /identity/v3/projects => generated 365 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:30:53.675105 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c6b4de-9e72-4089-86df-ca372f9b4392 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:30:53.685486 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c6b4de-9e72-4089-86df-ca372f9b4392 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:30:53.749696 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70613a89-4cca-46d6-bccc-93817096b78e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:30:53.750375 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70613a89-4cca-46d6-bccc-93817096b78e admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:30:53.750650 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70613a89-4cca-46d6-bccc-93817096b78e admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:30:53.752462 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70613a89-4cca-46d6-bccc-93817096b78e admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:30:53.752462 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70613a89-4cca-46d6-bccc-93817096b78e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:30:53.754104 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70613a89-4cca-46d6-bccc-93817096b78e admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:30:53.822931 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1130/2285] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:30:53 2022] POST /identity/v3/users => generated 283 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:30:53.827289 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491f317c-cdfc-41f1-bc86-4dfc15d338a6 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:30:53.840700 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491f317c-cdfc-41f1-bc86-4dfc15d338a6 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:30:53.845969 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b297a3e-0d29-45a1-a0f7-970be936b81d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:30:53.846733 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b297a3e-0d29-45a1-a0f7-970be936b81d admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:30:53.847204 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b297a3e-0d29-45a1-a0f7-970be936b81d admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:30:53.847533 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b297a3e-0d29-45a1-a0f7-970be936b81d admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:30:53.848244 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b297a3e-0d29-45a1-a0f7-970be936b81d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:30:53.849892 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b297a3e-0d29-45a1-a0f7-970be936b81d admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:30:53.862054 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1156/2286] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:30:53 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:30:53.866228 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70613a89-4cca-46d6-bccc-93817096b78e admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:30:53.877986 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70613a89-4cca-46d6-bccc-93817096b78e admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:30:53.882844 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb3e3a8-260f-4707-b16f-aac2ddeebc02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:30:53.883501 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb3e3a8-260f-4707-b16f-aac2ddeebc02 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:30:53.883786 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb3e3a8-260f-4707-b16f-aac2ddeebc02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:30:53.884058 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb3e3a8-260f-4707-b16f-aac2ddeebc02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:30:53.884760 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb3e3a8-260f-4707-b16f-aac2ddeebc02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:30:53.886369 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb3e3a8-260f-4707-b16f-aac2ddeebc02 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:30:53.897968 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1131/2287] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:30:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:30:53.903122 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b297a3e-0d29-45a1-a0f7-970be936b81d admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:30:53.917022 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b297a3e-0d29-45a1-a0f7-970be936b81d admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:30:53.922313 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899174f5-27ee-4a2d-8232-4943849f4975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:30:53.923730 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899174f5-27ee-4a2d-8232-4943849f4975 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:30:53.924095 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899174f5-27ee-4a2d-8232-4943849f4975 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:30:53.924459 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899174f5-27ee-4a2d-8232-4943849f4975 admin admin] PATH_INFO: `/v3/projects/2815f35a1add474cb3c5f130c768f3d0/users/35c0845cb6064aac958f726ab179d4c2/roles/8feebc60363c4feb870718b1721e5324` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:30:53.933608 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:30:53.941667 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899174f5-27ee-4a2d-8232-4943849f4975 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2815f35a1add474cb3c5f130c768f3d0, user_id=35c0845cb6064aac958f726ab179d4c2, role_id=8feebc60363c4feb870718b1721e5324)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:30:53.945148 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899174f5-27ee-4a2d-8232-4943849f4975 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:30:54.029995 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1157/2288] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:30:53 2022] PUT /identity/v3/projects/2815f35a1add474cb3c5f130c768f3d0/users/35c0845cb6064aac958f726ab179d4c2/roles/8feebc60363c4feb870718b1721e5324 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:30:54.038460 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad29e87-af8b-4591-ac30-029957a01acf None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:30:54.038826 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad29e87-af8b-4591-ac30-029957a01acf None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:30:54.039169 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad29e87-af8b-4591-ac30-029957a01acf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:30:54.067922 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-bad29e87-af8b-4591-ac30-029957a01acf None None] MFA Rules not processed for user `35c0845cb6064aac958f726ab179d4c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:30:54.069438 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-bad29e87-af8b-4591-ac30-029957a01acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:30:54.137472 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1132/2289] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:30:54 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:30:54.148196 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899174f5-27ee-4a2d-8232-4943849f4975 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:30:54.149830 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-899174f5-27ee-4a2d-8232-4943849f4975 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:30:54.206826 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899174f5-27ee-4a2d-8232-4943849f4975 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:30:54.210499 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ab598a-1552-4c2c-8b6b-08f1b0dfc869 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e31f2728056b460fa474478eb85ca1d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:30:54.211115 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab598a-1552-4c2c-8b6b-08f1b0dfc869 service cinder] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:30:54.211435 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab598a-1552-4c2c-8b6b-08f1b0dfc869 service cinder] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:30:54.211716 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab598a-1552-4c2c-8b6b-08f1b0dfc869 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:30:54.220938 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ab598a-1552-4c2c-8b6b-08f1b0dfc869 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:30:54.222382 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ab598a-1552-4c2c-8b6b-08f1b0dfc869 service cinder] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:30:54.233203 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1158/2290] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:30:54 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:30:55.273660 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81049a5-9e63-40fb-89b1-eabc9a4905c1 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:30:55.278387 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81049a5-9e63-40fb-89b1-eabc9a4905c1 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:30:55.278387 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81049a5-9e63-40fb-89b1-eabc9a4905c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:30:55.294112 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-d81049a5-9e63-40fb-89b1-eabc9a4905c1 None None] MFA Rules not processed for user `23ea380385f649628cc27e3603bc700c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:30:55.295309 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-d81049a5-9e63-40fb-89b1-eabc9a4905c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:30:55.340248 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:30:55.351958 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1133/2291] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Aug 4 21:30:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:30:55.492306 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ab598a-1552-4c2c-8b6b-08f1b0dfc869 service cinder] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:30:55.493764 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-23ab598a-1552-4c2c-8b6b-08f1b0dfc869 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:30:55.571401 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ab598a-1552-4c2c-8b6b-08f1b0dfc869 service cinder] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:30:55.576898 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acef03b5-8dab-4245-b0f5-3f533d96a291 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58935367c0ad445f8477cdac0df04d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:30:55.577686 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acef03b5-8dab-4245-b0f5-3f533d96a291 service swift] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:30:55.578040 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acef03b5-8dab-4245-b0f5-3f533d96a291 service swift] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:30:55.578314 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acef03b5-8dab-4245-b0f5-3f533d96a291 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:30:55.588102 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acef03b5-8dab-4245-b0f5-3f533d96a291 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:30:55.589757 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acef03b5-8dab-4245-b0f5-3f533d96a291 service swift] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:30:55.600919 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1159/2292] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Aug 4 21:30:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 21:30:55.733427 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81049a5-9e63-40fb-89b1-eabc9a4905c1 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:30:55.736859 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-d81049a5-9e63-40fb-89b1-eabc9a4905c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:30:55.848159 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81049a5-9e63-40fb-89b1-eabc9a4905c1 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:30:55.865991 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897306fe-3768-4a5f-9103-50f55b778298 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a2ce1df87fa47e5be301916bf3245fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:30:55.866566 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897306fe-3768-4a5f-9103-50f55b778298 service nova] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:30:55.866848 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897306fe-3768-4a5f-9103-50f55b778298 service nova] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:30:55.867131 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897306fe-3768-4a5f-9103-50f55b778298 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:30:55.869239 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-897306fe-3768-4a5f-9103-50f55b778298 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:30:55.900488 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897306fe-3768-4a5f-9103-50f55b778298 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:30:55.903360 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897306fe-3768-4a5f-9103-50f55b778298 service nova] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:30:55.926860 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1134/2293] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:30:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:31:02.222252 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acef03b5-8dab-4245-b0f5-3f533d96a291 service swift] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:31:02.237086 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acef03b5-8dab-4245-b0f5-3f533d96a291 service swift] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:31:02.260779 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d046b5-fd81-4c59-9fa8-f882c50c6721 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a2ce1df87fa47e5be301916bf3245fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:31:02.261423 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d046b5-fd81-4c59-9fa8-f882c50c6721 service nova] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:02.261708 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d046b5-fd81-4c59-9fa8-f882c50c6721 service nova] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:02.262033 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d046b5-fd81-4c59-9fa8-f882c50c6721 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:02.265210 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-43d046b5-fd81-4c59-9fa8-f882c50c6721 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:31:02.352816 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d046b5-fd81-4c59-9fa8-f882c50c6721 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:31:02.354446 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d046b5-fd81-4c59-9fa8-f882c50c6721 service nova] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:31:02.373278 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1160/2294] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:31:02 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:31:04.906636 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897306fe-3768-4a5f-9103-50f55b778298 service nova] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:31:04.996346 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897306fe-3768-4a5f-9103-50f55b778298 service nova] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:31:05.002972 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ce47dc-dc03-40d5-9de7-79926f6399d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:31:05.003726 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce47dc-dc03-40d5-9de7-79926f6399d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:05.004003 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce47dc-dc03-40d5-9de7-79926f6399d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:05.004272 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce47dc-dc03-40d5-9de7-79926f6399d3 admin admin] PATH_INFO: `/v3/users/35c0845cb6064aac958f726ab179d4c2` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:05.005456 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ce47dc-dc03-40d5-9de7-79926f6399d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35c0845cb6064aac958f726ab179d4c2)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:31:05.007479 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ce47dc-dc03-40d5-9de7-79926f6399d3 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:31:05.217653 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-57ce47dc-dc03-40d5-9de7-79926f6399d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35c0845cb6064aac958f726ab179d4c2'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:31:05.249366 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-57ce47dc-dc03-40d5-9de7-79926f6399d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35c0845cb6064aac958f726ab179d4c2'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:31:05.250137 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-57ce47dc-dc03-40d5-9de7-79926f6399d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35c0845cb6064aac958f726ab179d4c2'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:31:05.259570 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-57ce47dc-dc03-40d5-9de7-79926f6399d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35c0845cb6064aac958f726ab179d4c2'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:31:05.261720 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-57ce47dc-dc03-40d5-9de7-79926f6399d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35c0845cb6064aac958f726ab179d4c2'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:31:05.356117 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:31:05.366262 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1135/2295] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:31:04 2022] DELETE /identity/v3/users/35c0845cb6064aac958f726ab179d4c2 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:31:05.373463 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b81456-4086-491c-9152-e90ca6ddd9db None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:05.373463 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b81456-4086-491c-9152-e90ca6ddd9db None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:05.373463 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b81456-4086-491c-9152-e90ca6ddd9db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:05.390093 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-12b81456-4086-491c-9152-e90ca6ddd9db None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:31:05.391345 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-12b81456-4086-491c-9152-e90ca6ddd9db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:31:05.404544 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:31:05.459419 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1161/2296] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:31:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:31:05.468170 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ce47dc-dc03-40d5-9de7-79926f6399d3 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:31:05.469888 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-57ce47dc-dc03-40d5-9de7-79926f6399d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:31:05.536127 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ce47dc-dc03-40d5-9de7-79926f6399d3 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:31:05.541135 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b72f75-ff2d-4ed0-a754-a479580fc4c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:31:05.541929 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b72f75-ff2d-4ed0-a754-a479580fc4c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:05.542216 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b72f75-ff2d-4ed0-a754-a479580fc4c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:05.542484 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b72f75-ff2d-4ed0-a754-a479580fc4c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:05.554536 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b72f75-ff2d-4ed0-a754-a479580fc4c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:31:05.556275 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b72f75-ff2d-4ed0-a754-a479580fc4c3 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:31:05.567044 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1136/2297] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:31:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:31:06.047441 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b81456-4086-491c-9152-e90ca6ddd9db None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:31:06.049619 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-12b81456-4086-491c-9152-e90ca6ddd9db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:31:06.069671 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b81456-4086-491c-9152-e90ca6ddd9db None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:31:06.069671 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d1fffd-ec04-42cb-818f-0c8c381c3d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:31:06.069671 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d1fffd-ec04-42cb-818f-0c8c381c3d50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:06.069671 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d1fffd-ec04-42cb-818f-0c8c381c3d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:06.070421 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d1fffd-ec04-42cb-818f-0c8c381c3d50 admin admin] PATH_INFO: `/v3/projects/2815f35a1add474cb3c5f130c768f3d0` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:06.071924 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d1fffd-ec04-42cb-818f-0c8c381c3d50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2815f35a1add474cb3c5f130c768f3d0)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:31:06.073926 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d1fffd-ec04-42cb-818f-0c8c381c3d50 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:31:06.208288 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-42d1fffd-ec04-42cb-818f-0c8c381c3d50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2815f35a1add474cb3c5f130c768f3d0', 'request_id': 'req-42d1fffd-ec04-42cb-818f-0c8c381c3d50'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:31:06.217921 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-42d1fffd-ec04-42cb-818f-0c8c381c3d50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2815f35a1add474cb3c5f130c768f3d0', 'request_id': 'req-42d1fffd-ec04-42cb-818f-0c8c381c3d50'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:31:06.294245 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-42d1fffd-ec04-42cb-818f-0c8c381c3d50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2815f35a1add474cb3c5f130c768f3d0', 'request_id': 'req-42d1fffd-ec04-42cb-818f-0c8c381c3d50'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:31:06.321428 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-42d1fffd-ec04-42cb-818f-0c8c381c3d50 admin admin] The token cache is being invalidate because project 2815f35a1add474cb3c5f130c768f3d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:31:06.322516 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-42d1fffd-ec04-42cb-818f-0c8c381c3d50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:31:06.323173 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-42d1fffd-ec04-42cb-818f-0c8c381c3d50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:31:06.325444 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1162/2298] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:31:06 2022] DELETE /identity/v3/projects/2815f35a1add474cb3c5f130c768f3d0 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:31:06.336260 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70ceffe-99c8-4b44-9258-7230f33e39fb None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:06.337060 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70ceffe-99c8-4b44-9258-7230f33e39fb None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:06.337060 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70ceffe-99c8-4b44-9258-7230f33e39fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:06.366902 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-e70ceffe-99c8-4b44-9258-7230f33e39fb None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:31:06.368152 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-e70ceffe-99c8-4b44-9258-7230f33e39fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:31:06.435150 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1137/2299] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:31:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:31:06.443147 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70607cab-4152-4d1b-9046-45a45e9ec307 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:06.444637 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70607cab-4152-4d1b-9046-45a45e9ec307 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:06.444637 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70607cab-4152-4d1b-9046-45a45e9ec307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:06.465175 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-70607cab-4152-4d1b-9046-45a45e9ec307 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:31:06.467629 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-70607cab-4152-4d1b-9046-45a45e9ec307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:31:06.475790 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1163/2300] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:31:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:31:06.480241 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70ceffe-99c8-4b44-9258-7230f33e39fb None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:31:06.494458 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70ceffe-99c8-4b44-9258-7230f33e39fb None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:31:06.499427 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaf94bb-629a-4a35-ba93-5d03085fefe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:31:06.500597 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf94bb-629a-4a35-ba93-5d03085fefe8 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:06.500597 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf94bb-629a-4a35-ba93-5d03085fefe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:06.501090 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf94bb-629a-4a35-ba93-5d03085fefe8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:06.501494 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaf94bb-629a-4a35-ba93-5d03085fefe8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:31:06.501818 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaf94bb-629a-4a35-ba93-5d03085fefe8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:31:06.503742 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaf94bb-629a-4a35-ba93-5d03085fefe8 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:31:06.528343 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1138/2301] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Aug 4 21:31:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:31:06.533821 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70607cab-4152-4d1b-9046-45a45e9ec307 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:31:06.544540 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70607cab-4152-4d1b-9046-45a45e9ec307 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:31:06.549294 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899f1be0-8330-435c-a154-70b19bfdd7fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:31:06.549916 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899f1be0-8330-435c-a154-70b19bfdd7fe admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:06.550203 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899f1be0-8330-435c-a154-70b19bfdd7fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:06.550478 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899f1be0-8330-435c-a154-70b19bfdd7fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:06.551152 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899f1be0-8330-435c-a154-70b19bfdd7fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:31:06.553112 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899f1be0-8330-435c-a154-70b19bfdd7fe admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:31:06.638204 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1164/2302] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:31:06 2022] POST /identity/v3/projects => generated 401 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:31:06.642997 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaf94bb-629a-4a35-ba93-5d03085fefe8 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:31:06.653525 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaf94bb-629a-4a35-ba93-5d03085fefe8 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:31:06.718888 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dab1e92-8c1e-41c2-be94-26a4f0aca9a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:31:06.719557 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dab1e92-8c1e-41c2-be94-26a4f0aca9a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:06.719835 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dab1e92-8c1e-41c2-be94-26a4f0aca9a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:06.720098 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dab1e92-8c1e-41c2-be94-26a4f0aca9a7 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:06.720797 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dab1e92-8c1e-41c2-be94-26a4f0aca9a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:31:06.722740 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dab1e92-8c1e-41c2-be94-26a4f0aca9a7 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:31:06.851270 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1139/2303] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:31:06 2022] POST /identity/v3/users => generated 301 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:31:06.855307 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899f1be0-8330-435c-a154-70b19bfdd7fe admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:31:06.866159 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899f1be0-8330-435c-a154-70b19bfdd7fe admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:31:06.871443 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cd4742-6534-4d26-bfc7-db7221fc41d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:31:06.872297 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cd4742-6534-4d26-bfc7-db7221fc41d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:06.872857 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cd4742-6534-4d26-bfc7-db7221fc41d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:06.873133 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cd4742-6534-4d26-bfc7-db7221fc41d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:06.873831 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cd4742-6534-4d26-bfc7-db7221fc41d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:31:06.875465 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cd4742-6534-4d26-bfc7-db7221fc41d1 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:31:06.887493 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1165/2304] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:31:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:31:06.893004 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dab1e92-8c1e-41c2-be94-26a4f0aca9a7 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:31:06.904268 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dab1e92-8c1e-41c2-be94-26a4f0aca9a7 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:31:06.909269 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bcd2c4-381f-4b0c-a39c-a13bf09c0a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:31:06.909894 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bcd2c4-381f-4b0c-a39c-a13bf09c0a20 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:06.910177 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bcd2c4-381f-4b0c-a39c-a13bf09c0a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:06.910451 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bcd2c4-381f-4b0c-a39c-a13bf09c0a20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:06.911154 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bcd2c4-381f-4b0c-a39c-a13bf09c0a20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:31:06.912900 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bcd2c4-381f-4b0c-a39c-a13bf09c0a20 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:31:06.924264 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1140/2305] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:31:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:31:06.928581 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cd4742-6534-4d26-bfc7-db7221fc41d1 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:31:06.941235 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cd4742-6534-4d26-bfc7-db7221fc41d1 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:31:06.947149 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95106937-8982-4498-9b20-e13bb69e9da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:31:06.948150 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95106937-8982-4498-9b20-e13bb69e9da2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:06.948608 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95106937-8982-4498-9b20-e13bb69e9da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:06.948888 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95106937-8982-4498-9b20-e13bb69e9da2 admin admin] PATH_INFO: `/v3/projects/efe380fc54b140e5bf6dd05802b2586d/users/d4dce4c8c8de4204b9faf124fc1fe64e/roles/8feebc60363c4feb870718b1721e5324` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:06.965337 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95106937-8982-4498-9b20-e13bb69e9da2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efe380fc54b140e5bf6dd05802b2586d, user_id=d4dce4c8c8de4204b9faf124fc1fe64e, role_id=8feebc60363c4feb870718b1721e5324)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:31:06.968042 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95106937-8982-4498-9b20-e13bb69e9da2 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:31:07.070394 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1166/2306] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:31:06 2022] PUT /identity/v3/projects/efe380fc54b140e5bf6dd05802b2586d/users/d4dce4c8c8de4204b9faf124fc1fe64e/roles/8feebc60363c4feb870718b1721e5324 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:31:07.081762 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a58a75b-edf0-4d1e-ad6f-183141255487 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:07.082303 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a58a75b-edf0-4d1e-ad6f-183141255487 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:07.082588 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a58a75b-edf0-4d1e-ad6f-183141255487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:07.096843 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:31:07.112555 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-4a58a75b-edf0-4d1e-ad6f-183141255487 None None] MFA Rules not processed for user `d4dce4c8c8de4204b9faf124fc1fe64e`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:31:07.113684 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-4a58a75b-edf0-4d1e-ad6f-183141255487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:31:07.188316 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1141/2307] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:31:07 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:31:07.192554 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95106937-8982-4498-9b20-e13bb69e9da2 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:31:07.241721 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:31:07.265830 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95106937-8982-4498-9b20-e13bb69e9da2 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:31:07.270621 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c345092-6d6f-49f8-bd4e-60663bb4faec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:31:07.271243 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c345092-6d6f-49f8-bd4e-60663bb4faec admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:07.271637 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c345092-6d6f-49f8-bd4e-60663bb4faec admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:07.271843 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c345092-6d6f-49f8-bd4e-60663bb4faec admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:07.272657 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c345092-6d6f-49f8-bd4e-60663bb4faec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:31:07.274579 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c345092-6d6f-49f8-bd4e-60663bb4faec admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:31:07.375007 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1167/2308] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:31:07 2022] POST /identity/v3/projects => generated 399 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:31:07.379740 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a58a75b-edf0-4d1e-ad6f-183141255487 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:31:07.390460 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a58a75b-edf0-4d1e-ad6f-183141255487 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:31:07.451662 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c298a6df-6b7e-41f3-bfc0-ecdafbc15daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:31:07.452578 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c298a6df-6b7e-41f3-bfc0-ecdafbc15daf admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:07.452956 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c298a6df-6b7e-41f3-bfc0-ecdafbc15daf admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:07.453302 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c298a6df-6b7e-41f3-bfc0-ecdafbc15daf admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:07.454116 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c298a6df-6b7e-41f3-bfc0-ecdafbc15daf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:31:07.456481 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c298a6df-6b7e-41f3-bfc0-ecdafbc15daf admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:31:07.541369 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1142/2309] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:31:07 2022] POST /identity/v3/users => generated 306 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:31:07.545613 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c345092-6d6f-49f8-bd4e-60663bb4faec admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:31:07.556948 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c345092-6d6f-49f8-bd4e-60663bb4faec admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:31:07.561649 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ab0c30-d29d-448b-82be-b79cc9c5fd85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:31:07.562265 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ab0c30-d29d-448b-82be-b79cc9c5fd85 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:07.562551 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ab0c30-d29d-448b-82be-b79cc9c5fd85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:07.562830 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ab0c30-d29d-448b-82be-b79cc9c5fd85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:07.563563 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ab0c30-d29d-448b-82be-b79cc9c5fd85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:31:07.565174 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ab0c30-d29d-448b-82be-b79cc9c5fd85 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:31:07.575495 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1168/2310] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:31:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:31:07.579765 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c298a6df-6b7e-41f3-bfc0-ecdafbc15daf admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:31:07.590128 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c298a6df-6b7e-41f3-bfc0-ecdafbc15daf admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:31:07.595295 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d134262a-5cf6-4b66-b9ed-fbb08223c699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:31:07.595980 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d134262a-5cf6-4b66-b9ed-fbb08223c699 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:07.596286 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d134262a-5cf6-4b66-b9ed-fbb08223c699 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:07.596552 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d134262a-5cf6-4b66-b9ed-fbb08223c699 admin admin] PATH_INFO: `/v3/projects/c69a13f54a584b52b99cc0ec0f928980/users/0971546346ab48f2993ea6523f0d3dfb/roles/a432dd4735944ecfab2fa5ce1705d356` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:07.612946 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d134262a-5cf6-4b66-b9ed-fbb08223c699 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c69a13f54a584b52b99cc0ec0f928980, user_id=0971546346ab48f2993ea6523f0d3dfb, role_id=a432dd4735944ecfab2fa5ce1705d356)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:31:07.615658 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d134262a-5cf6-4b66-b9ed-fbb08223c699 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:31:07.687807 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1143/2311] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:31:07 2022] PUT /identity/v3/projects/c69a13f54a584b52b99cc0ec0f928980/users/0971546346ab48f2993ea6523f0d3dfb/roles/a432dd4735944ecfab2fa5ce1705d356 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:31:07.692047 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ab0c30-d29d-448b-82be-b79cc9c5fd85 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:31:07.703088 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ab0c30-d29d-448b-82be-b79cc9c5fd85 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:31:07.761962 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcd3f8e-ee78-43bc-bf9b-a994b977ac9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:31:07.762578 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcd3f8e-ee78-43bc-bf9b-a994b977ac9f admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:07.762858 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcd3f8e-ee78-43bc-bf9b-a994b977ac9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:07.763129 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcd3f8e-ee78-43bc-bf9b-a994b977ac9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:07.763855 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcd3f8e-ee78-43bc-bf9b-a994b977ac9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:31:07.765443 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcd3f8e-ee78-43bc-bf9b-a994b977ac9f admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:31:07.775418 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1169/2312] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:31:07 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:31:07.779709 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d134262a-5cf6-4b66-b9ed-fbb08223c699 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:31:07.790552 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d134262a-5cf6-4b66-b9ed-fbb08223c699 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:31:07.795383 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d595c9-6585-45c3-ab21-c166abbcaef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:31:07.796137 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d595c9-6585-45c3-ab21-c166abbcaef7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:07.796331 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d595c9-6585-45c3-ab21-c166abbcaef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:07.796739 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d595c9-6585-45c3-ab21-c166abbcaef7 admin admin] PATH_INFO: `/v3/projects/c69a13f54a584b52b99cc0ec0f928980/users/0971546346ab48f2993ea6523f0d3dfb/roles/a432dd4735944ecfab2fa5ce1705d356` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:07.799310 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d595c9-6585-45c3-ab21-c166abbcaef7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c69a13f54a584b52b99cc0ec0f928980, user_id=0971546346ab48f2993ea6523f0d3dfb, role_id=a432dd4735944ecfab2fa5ce1705d356)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:31:07.801998 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d595c9-6585-45c3-ab21-c166abbcaef7 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:31:07.812387 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1144/2313] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:31:07 2022] PUT /identity/v3/projects/c69a13f54a584b52b99cc0ec0f928980/users/0971546346ab48f2993ea6523f0d3dfb/roles/a432dd4735944ecfab2fa5ce1705d356 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:31:07.824541 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb08e5c-1d5a-475c-8efa-e76d9e3b4b3c None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:07.824832 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb08e5c-1d5a-475c-8efa-e76d9e3b4b3c None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:07.825135 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb08e5c-1d5a-475c-8efa-e76d9e3b4b3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:07.854306 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-0eb08e5c-1d5a-475c-8efa-e76d9e3b4b3c None None] MFA Rules not processed for user `0971546346ab48f2993ea6523f0d3dfb`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:31:07.855565 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-0eb08e5c-1d5a-475c-8efa-e76d9e3b4b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:31:07.927138 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1170/2314] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:31:07 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:31:07.938317 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d595c9-6585-45c3-ab21-c166abbcaef7 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:31:07.939826 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-88d595c9-6585-45c3-ab21-c166abbcaef7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:31:07.994520 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:31:07.998339 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d595c9-6585-45c3-ab21-c166abbcaef7 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:31:08.002187 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b213cd-2bb0-45f3-aa0b-8ee59f4d50f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e31f2728056b460fa474478eb85ca1d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:31:08.002817 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b213cd-2bb0-45f3-aa0b-8ee59f4d50f9 service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:08.006512 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b213cd-2bb0-45f3-aa0b-8ee59f4d50f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:08.006512 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b213cd-2bb0-45f3-aa0b-8ee59f4d50f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:08.013301 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b213cd-2bb0-45f3-aa0b-8ee59f4d50f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:31:08.014752 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b213cd-2bb0-45f3-aa0b-8ee59f4d50f9 service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:31:08.025485 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1145/2315] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:31:07 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:31:08.218219 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb08e5c-1d5a-475c-8efa-e76d9e3b4b3c None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:31:08.228403 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb08e5c-1d5a-475c-8efa-e76d9e3b4b3c None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:31:08.231981 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13169ebf-8927-4d98-b6bb-9b1d7d2beb71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e31f2728056b460fa474478eb85ca1d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:31:08.232593 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13169ebf-8927-4d98-b6bb-9b1d7d2beb71 service cinder] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:08.232872 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13169ebf-8927-4d98-b6bb-9b1d7d2beb71 service cinder] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:08.233141 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13169ebf-8927-4d98-b6bb-9b1d7d2beb71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:08.264620 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:31:08.292597 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13169ebf-8927-4d98-b6bb-9b1d7d2beb71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:31:08.294055 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13169ebf-8927-4d98-b6bb-9b1d7d2beb71 service cinder] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:31:08.303070 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1171/2316] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:31:08 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:31:13.607782 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b213cd-2bb0-45f3-aa0b-8ee59f4d50f9 service cinder] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:31:13.690145 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b213cd-2bb0-45f3-aa0b-8ee59f4d50f9 service cinder] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:31:13.695100 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e2a6a2-b35e-4bf7-88a5-3de682618bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:31:13.695793 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e2a6a2-b35e-4bf7-88a5-3de682618bb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:13.696075 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e2a6a2-b35e-4bf7-88a5-3de682618bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:13.696351 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e2a6a2-b35e-4bf7-88a5-3de682618bb7 admin admin] PATH_INFO: `/v3/users/d4dce4c8c8de4204b9faf124fc1fe64e` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:13.697509 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e2a6a2-b35e-4bf7-88a5-3de682618bb7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4dce4c8c8de4204b9faf124fc1fe64e)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:31:13.699493 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e2a6a2-b35e-4bf7-88a5-3de682618bb7 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:31:13.886250 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-22e2a6a2-b35e-4bf7-88a5-3de682618bb7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4dce4c8c8de4204b9faf124fc1fe64e'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:31:13.899807 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-22e2a6a2-b35e-4bf7-88a5-3de682618bb7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4dce4c8c8de4204b9faf124fc1fe64e'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:31:13.900554 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-22e2a6a2-b35e-4bf7-88a5-3de682618bb7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4dce4c8c8de4204b9faf124fc1fe64e'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:31:13.908152 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-22e2a6a2-b35e-4bf7-88a5-3de682618bb7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4dce4c8c8de4204b9faf124fc1fe64e'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:31:13.908854 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-22e2a6a2-b35e-4bf7-88a5-3de682618bb7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4dce4c8c8de4204b9faf124fc1fe64e'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:31:13.994444 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1146/2317] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:31:13 2022] DELETE /identity/v3/users/d4dce4c8c8de4204b9faf124fc1fe64e => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:31:13.998957 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7093bacb-3d86-416b-87a0-c24d6c190990 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:13.999531 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7093bacb-3d86-416b-87a0-c24d6c190990 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:13.999627 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7093bacb-3d86-416b-87a0-c24d6c190990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:14.021119 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-7093bacb-3d86-416b-87a0-c24d6c190990 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:31:14.022333 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-7093bacb-3d86-416b-87a0-c24d6c190990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:31:14.083166 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1172/2318] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:31:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:31:14.093414 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e2a6a2-b35e-4bf7-88a5-3de682618bb7 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:31:14.095000 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-22e2a6a2-b35e-4bf7-88a5-3de682618bb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:31:14.154576 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e2a6a2-b35e-4bf7-88a5-3de682618bb7 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:31:14.158168 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5082bece-d3ad-45a4-b066-9f9048f4289b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:31:14.158773 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5082bece-d3ad-45a4-b066-9f9048f4289b service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:14.159050 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5082bece-d3ad-45a4-b066-9f9048f4289b service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:14.159589 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5082bece-d3ad-45a4-b066-9f9048f4289b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:14.169109 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5082bece-d3ad-45a4-b066-9f9048f4289b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:31:14.171057 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5082bece-d3ad-45a4-b066-9f9048f4289b service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:31:14.184593 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1147/2319] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:31:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:31:14.859667 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7093bacb-3d86-416b-87a0-c24d6c190990 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:31:14.861155 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-7093bacb-3d86-416b-87a0-c24d6c190990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:31:14.873106 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7093bacb-3d86-416b-87a0-c24d6c190990 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:31:14.877937 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a7f289-4ddb-41cc-a75d-d324eba8e0dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:31:14.878555 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a7f289-4ddb-41cc-a75d-d324eba8e0dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:14.878821 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a7f289-4ddb-41cc-a75d-d324eba8e0dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:14.879101 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a7f289-4ddb-41cc-a75d-d324eba8e0dd admin admin] PATH_INFO: `/v3/projects/efe380fc54b140e5bf6dd05802b2586d` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:14.880532 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a7f289-4ddb-41cc-a75d-d324eba8e0dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=efe380fc54b140e5bf6dd05802b2586d)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:31:14.882356 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a7f289-4ddb-41cc-a75d-d324eba8e0dd admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:31:15.026292 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-19a7f289-4ddb-41cc-a75d-d324eba8e0dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efe380fc54b140e5bf6dd05802b2586d', 'request_id': 'req-19a7f289-4ddb-41cc-a75d-d324eba8e0dd'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:31:15.031861 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-19a7f289-4ddb-41cc-a75d-d324eba8e0dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efe380fc54b140e5bf6dd05802b2586d', 'request_id': 'req-19a7f289-4ddb-41cc-a75d-d324eba8e0dd'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:31:15.108093 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-19a7f289-4ddb-41cc-a75d-d324eba8e0dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efe380fc54b140e5bf6dd05802b2586d', 'request_id': 'req-19a7f289-4ddb-41cc-a75d-d324eba8e0dd'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:31:15.119597 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-19a7f289-4ddb-41cc-a75d-d324eba8e0dd admin admin] The token cache is being invalidate because project efe380fc54b140e5bf6dd05802b2586d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:31:15.122553 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-19a7f289-4ddb-41cc-a75d-d324eba8e0dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:31:15.122553 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-19a7f289-4ddb-41cc-a75d-d324eba8e0dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:31:15.122553 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1173/2320] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:31:14 2022] DELETE /identity/v3/projects/efe380fc54b140e5bf6dd05802b2586d => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:31:15.126563 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5082bece-d3ad-45a4-b066-9f9048f4289b service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:31:15.137569 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5082bece-d3ad-45a4-b066-9f9048f4289b service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:31:15.146046 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:31:15.196032 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93626c5b-833b-432a-8c18-d1b017e41392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:31:15.196684 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93626c5b-833b-432a-8c18-d1b017e41392 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:15.196961 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93626c5b-833b-432a-8c18-d1b017e41392 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:15.197233 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93626c5b-833b-432a-8c18-d1b017e41392 admin admin] PATH_INFO: `/v3/users/0971546346ab48f2993ea6523f0d3dfb` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:15.198404 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93626c5b-833b-432a-8c18-d1b017e41392 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0971546346ab48f2993ea6523f0d3dfb)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:31:15.200387 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93626c5b-833b-432a-8c18-d1b017e41392 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:31:15.403331 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507cd1bf-8847-484a-98f4-aa964aa76242 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:15.403643 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507cd1bf-8847-484a-98f4-aa964aa76242 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:15.403924 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507cd1bf-8847-484a-98f4-aa964aa76242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:15.423789 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-507cd1bf-8847-484a-98f4-aa964aa76242 None None] MFA Rules not processed for user `23ea380385f649628cc27e3603bc700c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:31:15.425006 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-507cd1bf-8847-484a-98f4-aa964aa76242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:31:15.450340 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-93626c5b-833b-432a-8c18-d1b017e41392 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0971546346ab48f2993ea6523f0d3dfb'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:31:15.464280 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-93626c5b-833b-432a-8c18-d1b017e41392 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0971546346ab48f2993ea6523f0d3dfb'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:31:15.465174 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-93626c5b-833b-432a-8c18-d1b017e41392 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0971546346ab48f2993ea6523f0d3dfb'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:31:15.472798 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-93626c5b-833b-432a-8c18-d1b017e41392 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0971546346ab48f2993ea6523f0d3dfb'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:31:15.473465 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-93626c5b-833b-432a-8c18-d1b017e41392 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0971546346ab48f2993ea6523f0d3dfb'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:31:15.478811 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:31:15.486919 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1174/2321] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Aug 4 21:31:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:31:15.498928 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507cd1bf-8847-484a-98f4-aa964aa76242 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:31:15.500293 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-507cd1bf-8847-484a-98f4-aa964aa76242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:31:15.573969 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1148/2322] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:31:15 2022] DELETE /identity/v3/users/0971546346ab48f2993ea6523f0d3dfb => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:31:15.579176 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507cd1bf-8847-484a-98f4-aa964aa76242 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:31:15.667944 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50c0baf-ee4a-48e4-81c8-16385f26aaea service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58935367c0ad445f8477cdac0df04d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:31:15.668591 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50c0baf-ee4a-48e4-81c8-16385f26aaea service swift] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:15.668870 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50c0baf-ee4a-48e4-81c8-16385f26aaea service swift] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:15.669141 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50c0baf-ee4a-48e4-81c8-16385f26aaea service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:15.747995 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50c0baf-ee4a-48e4-81c8-16385f26aaea service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:31:15.750196 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50c0baf-ee4a-48e4-81c8-16385f26aaea service swift] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:31:15.759455 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1175/2323] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Aug 4 21:31:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 04 21:31:16.109445 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93626c5b-833b-432a-8c18-d1b017e41392 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:31:16.111908 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-93626c5b-833b-432a-8c18-d1b017e41392 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:31:16.201338 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93626c5b-833b-432a-8c18-d1b017e41392 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:31:16.208487 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9d282d-6dac-49e4-a82b-b5285a683a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:31:16.209222 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9d282d-6dac-49e4-a82b-b5285a683a19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:16.209652 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9d282d-6dac-49e4-a82b-b5285a683a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:16.209907 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9d282d-6dac-49e4-a82b-b5285a683a19 admin admin] PATH_INFO: `/v3/projects/c69a13f54a584b52b99cc0ec0f928980` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:16.211297 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9d282d-6dac-49e4-a82b-b5285a683a19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c69a13f54a584b52b99cc0ec0f928980)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:31:16.213248 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9d282d-6dac-49e4-a82b-b5285a683a19 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:31:16.334251 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:31:16.357755 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-be9d282d-6dac-49e4-a82b-b5285a683a19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c69a13f54a584b52b99cc0ec0f928980', 'request_id': 'req-be9d282d-6dac-49e4-a82b-b5285a683a19'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:31:16.371437 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-be9d282d-6dac-49e4-a82b-b5285a683a19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c69a13f54a584b52b99cc0ec0f928980', 'request_id': 'req-be9d282d-6dac-49e4-a82b-b5285a683a19'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:31:16.442473 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-be9d282d-6dac-49e4-a82b-b5285a683a19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c69a13f54a584b52b99cc0ec0f928980', 'request_id': 'req-be9d282d-6dac-49e4-a82b-b5285a683a19'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:31:16.457916 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-be9d282d-6dac-49e4-a82b-b5285a683a19 admin admin] The token cache is being invalidate because project c69a13f54a584b52b99cc0ec0f928980 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:31:16.458218 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-be9d282d-6dac-49e4-a82b-b5285a683a19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:31:16.459135 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-be9d282d-6dac-49e4-a82b-b5285a683a19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:31:16.461555 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1149/2324] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:31:16 2022] DELETE /identity/v3/projects/c69a13f54a584b52b99cc0ec0f928980 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:31:16.467092 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1059a9-7dea-47d7-ba92-e7e9a75013f2 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:16.473224 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1059a9-7dea-47d7-ba92-e7e9a75013f2 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:16.473224 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1059a9-7dea-47d7-ba92-e7e9a75013f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:16.490621 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-bd1059a9-7dea-47d7-ba92-e7e9a75013f2 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:31:16.492204 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-bd1059a9-7dea-47d7-ba92-e7e9a75013f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:31:16.549237 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:31:16.576596 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1176/2325] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:31:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:31:16.585658 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a05316d-0b30-4a84-8ec0-832a358b56f2 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:16.586845 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a05316d-0b30-4a84-8ec0-832a358b56f2 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:16.587136 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a05316d-0b30-4a84-8ec0-832a358b56f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:16.612982 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-0a05316d-0b30-4a84-8ec0-832a358b56f2 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:31:16.614254 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-0a05316d-0b30-4a84-8ec0-832a358b56f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:31:16.622544 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1150/2326] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:31:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:31:16.628066 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1059a9-7dea-47d7-ba92-e7e9a75013f2 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:31:16.639823 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1059a9-7dea-47d7-ba92-e7e9a75013f2 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:31:16.647231 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c451eb56-80dd-45f2-855b-cd5c8472153e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:31:16.648131 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c451eb56-80dd-45f2-855b-cd5c8472153e admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:16.648524 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c451eb56-80dd-45f2-855b-cd5c8472153e admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:16.648864 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c451eb56-80dd-45f2-855b-cd5c8472153e admin admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:16.649663 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c451eb56-80dd-45f2-855b-cd5c8472153e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:31:16.650034 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c451eb56-80dd-45f2-855b-cd5c8472153e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:31:16.651783 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c451eb56-80dd-45f2-855b-cd5c8472153e admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:31:16.668112 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1177/2327] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Aug 4 21:31:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:31:16.672285 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a05316d-0b30-4a84-8ec0-832a358b56f2 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:31:16.684703 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a05316d-0b30-4a84-8ec0-832a358b56f2 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:31:16.691036 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c745a6a5-b450-4414-9b5a-71a286e0a2dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:31:16.691666 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c745a6a5-b450-4414-9b5a-71a286e0a2dc admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:16.692063 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c745a6a5-b450-4414-9b5a-71a286e0a2dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:16.692217 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c745a6a5-b450-4414-9b5a-71a286e0a2dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:16.693343 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c745a6a5-b450-4414-9b5a-71a286e0a2dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:31:16.695246 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c745a6a5-b450-4414-9b5a-71a286e0a2dc admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:31:16.794609 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1151/2328] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:31:16 2022] POST /identity/v3/projects => generated 369 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:31:16.801156 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c451eb56-80dd-45f2-855b-cd5c8472153e admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:31:16.817248 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c451eb56-80dd-45f2-855b-cd5c8472153e admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:31:16.919613 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54100ec3-a9a9-4bf6-84e3-8129cdc7fa41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:31:16.939772 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54100ec3-a9a9-4bf6-84e3-8129cdc7fa41 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:16.939772 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54100ec3-a9a9-4bf6-84e3-8129cdc7fa41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:16.939772 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54100ec3-a9a9-4bf6-84e3-8129cdc7fa41 admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:16.939772 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54100ec3-a9a9-4bf6-84e3-8129cdc7fa41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:31:16.939772 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54100ec3-a9a9-4bf6-84e3-8129cdc7fa41 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:31:17.172123 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1178/2329] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:31:16 2022] POST /identity/v3/users => generated 285 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:31:17.177070 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c745a6a5-b450-4414-9b5a-71a286e0a2dc admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:31:17.203455 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c745a6a5-b450-4414-9b5a-71a286e0a2dc admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:31:17.210716 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde31c01-f19a-4689-b8d9-0c9ccdc2a5b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:31:17.211621 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde31c01-f19a-4689-b8d9-0c9ccdc2a5b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:17.212112 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde31c01-f19a-4689-b8d9-0c9ccdc2a5b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:17.212564 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde31c01-f19a-4689-b8d9-0c9ccdc2a5b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:17.213682 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde31c01-f19a-4689-b8d9-0c9ccdc2a5b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:31:17.216471 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde31c01-f19a-4689-b8d9-0c9ccdc2a5b3 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:31:17.255806 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1152/2330] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:31:17 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:31:17.265739 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54100ec3-a9a9-4bf6-84e3-8129cdc7fa41 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:31:17.279334 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54100ec3-a9a9-4bf6-84e3-8129cdc7fa41 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:31:17.286146 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d93220-cc68-44ca-8c2e-73423b52c113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:31:17.287055 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d93220-cc68-44ca-8c2e-73423b52c113 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:17.287381 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d93220-cc68-44ca-8c2e-73423b52c113 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:17.287657 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d93220-cc68-44ca-8c2e-73423b52c113 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:17.288508 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d93220-cc68-44ca-8c2e-73423b52c113 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:31:17.290398 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d93220-cc68-44ca-8c2e-73423b52c113 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:31:17.309789 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1179/2331] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:31:17 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:31:17.316494 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde31c01-f19a-4689-b8d9-0c9ccdc2a5b3 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:31:17.339739 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde31c01-f19a-4689-b8d9-0c9ccdc2a5b3 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:31:17.349769 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644e02a1-6d52-4a47-b1f2-7d688bf7ebc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:31:17.350960 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644e02a1-6d52-4a47-b1f2-7d688bf7ebc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:17.351654 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644e02a1-6d52-4a47-b1f2-7d688bf7ebc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:17.352179 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644e02a1-6d52-4a47-b1f2-7d688bf7ebc5 admin admin] PATH_INFO: `/v3/projects/df2cd6c23ca843a4a7a89dbc9e835157/users/b4fcc0cb80cd4a0ba270f0b106f65f33/roles/8feebc60363c4feb870718b1721e5324` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:17.383909 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644e02a1-6d52-4a47-b1f2-7d688bf7ebc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df2cd6c23ca843a4a7a89dbc9e835157, user_id=b4fcc0cb80cd4a0ba270f0b106f65f33, role_id=8feebc60363c4feb870718b1721e5324)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:31:17.390298 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644e02a1-6d52-4a47-b1f2-7d688bf7ebc5 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:31:17.510480 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1153/2332] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:31:17 2022] PUT /identity/v3/projects/df2cd6c23ca843a4a7a89dbc9e835157/users/b4fcc0cb80cd4a0ba270f0b106f65f33/roles/8feebc60363c4feb870718b1721e5324 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:31:17.513280 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d095fa5d-5229-4762-8607-31bca6ab3c4e None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:17.518568 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d095fa5d-5229-4762-8607-31bca6ab3c4e None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:17.518568 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d095fa5d-5229-4762-8607-31bca6ab3c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:17.545793 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-d095fa5d-5229-4762-8607-31bca6ab3c4e None None] MFA Rules not processed for user `b4fcc0cb80cd4a0ba270f0b106f65f33`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:31:17.547022 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-d095fa5d-5229-4762-8607-31bca6ab3c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:31:17.629047 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1180/2333] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:31:17 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:31:17.637960 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644e02a1-6d52-4a47-b1f2-7d688bf7ebc5 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:31:17.639995 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-644e02a1-6d52-4a47-b1f2-7d688bf7ebc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:31:17.715636 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644e02a1-6d52-4a47-b1f2-7d688bf7ebc5 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:31:17.719442 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfccd6d-ca6b-4169-a526-88bdaca9df39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e31f2728056b460fa474478eb85ca1d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:31:17.720642 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfccd6d-ca6b-4169-a526-88bdaca9df39 service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:17.720929 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfccd6d-ca6b-4169-a526-88bdaca9df39 service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:17.721240 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfccd6d-ca6b-4169-a526-88bdaca9df39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:17.731799 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfccd6d-ca6b-4169-a526-88bdaca9df39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:31:17.733274 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfccd6d-ca6b-4169-a526-88bdaca9df39 service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:31:17.742884 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1154/2334] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:31:17 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:31:17.778261 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d095fa5d-5229-4762-8607-31bca6ab3c4e None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:31:17.850265 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:31:17.856510 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d095fa5d-5229-4762-8607-31bca6ab3c4e None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:31:17.861176 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb51fdde-1fcd-4e67-a959-f68d35fd96b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:31:17.861812 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb51fdde-1fcd-4e67-a959-f68d35fd96b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:17.862095 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb51fdde-1fcd-4e67-a959-f68d35fd96b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:17.862373 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb51fdde-1fcd-4e67-a959-f68d35fd96b9 admin admin] PATH_INFO: `/v3/users/b4fcc0cb80cd4a0ba270f0b106f65f33` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:17.863587 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb51fdde-1fcd-4e67-a959-f68d35fd96b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4fcc0cb80cd4a0ba270f0b106f65f33)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:31:17.865508 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb51fdde-1fcd-4e67-a959-f68d35fd96b9 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:31:18.678048 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-fb51fdde-1fcd-4e67-a959-f68d35fd96b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4fcc0cb80cd4a0ba270f0b106f65f33'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:31:18.702557 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-fb51fdde-1fcd-4e67-a959-f68d35fd96b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4fcc0cb80cd4a0ba270f0b106f65f33'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:31:18.703506 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-fb51fdde-1fcd-4e67-a959-f68d35fd96b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4fcc0cb80cd4a0ba270f0b106f65f33'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:31:18.704260 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-fb51fdde-1fcd-4e67-a959-f68d35fd96b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4fcc0cb80cd4a0ba270f0b106f65f33'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:31:18.782498 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-fb51fdde-1fcd-4e67-a959-f68d35fd96b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4fcc0cb80cd4a0ba270f0b106f65f33'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:31:18.802965 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1181/2335] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:31:17 2022] DELETE /identity/v3/users/b4fcc0cb80cd4a0ba270f0b106f65f33 => generated 0 bytes in 1021 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:31:18.804785 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edebd21e-2aa0-402d-b40a-e8b0a82c25cf None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:18.805322 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edebd21e-2aa0-402d-b40a-e8b0a82c25cf None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:18.805611 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edebd21e-2aa0-402d-b40a-e8b0a82c25cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:18.828404 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-edebd21e-2aa0-402d-b40a-e8b0a82c25cf None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:31:18.829667 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-edebd21e-2aa0-402d-b40a-e8b0a82c25cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:31:18.909883 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1155/2336] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:31:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:31:18.917122 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb51fdde-1fcd-4e67-a959-f68d35fd96b9 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:31:18.918887 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-fb51fdde-1fcd-4e67-a959-f68d35fd96b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:31:19.010717 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb51fdde-1fcd-4e67-a959-f68d35fd96b9 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:31:19.014911 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326d61ce-0050-4c65-9ba4-5c2dfa344a12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:31:19.015655 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326d61ce-0050-4c65-9ba4-5c2dfa344a12 service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:19.015997 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326d61ce-0050-4c65-9ba4-5c2dfa344a12 service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:19.016419 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326d61ce-0050-4c65-9ba4-5c2dfa344a12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:19.029625 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326d61ce-0050-4c65-9ba4-5c2dfa344a12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:31:19.033053 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326d61ce-0050-4c65-9ba4-5c2dfa344a12 service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:31:19.043425 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1182/2337] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:31:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:31:19.590089 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edebd21e-2aa0-402d-b40a-e8b0a82c25cf None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:31:19.592065 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-edebd21e-2aa0-402d-b40a-e8b0a82c25cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:31:19.604130 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:31:19.611516 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edebd21e-2aa0-402d-b40a-e8b0a82c25cf None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:31:19.619620 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b911253-6737-45a9-93b5-2b1fc737889d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:31:19.621188 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b911253-6737-45a9-93b5-2b1fc737889d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:19.621886 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b911253-6737-45a9-93b5-2b1fc737889d admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:19.622522 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b911253-6737-45a9-93b5-2b1fc737889d admin admin] PATH_INFO: `/v3/projects/df2cd6c23ca843a4a7a89dbc9e835157` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:19.624825 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b911253-6737-45a9-93b5-2b1fc737889d admin admin] RBAC: Authorizing `identity:delete_project(project_id=df2cd6c23ca843a4a7a89dbc9e835157)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:31:19.629265 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b911253-6737-45a9-93b5-2b1fc737889d admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:31:19.742545 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-1b911253-6737-45a9-93b5-2b1fc737889d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df2cd6c23ca843a4a7a89dbc9e835157', 'request_id': 'req-1b911253-6737-45a9-93b5-2b1fc737889d'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:31:19.748715 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-1b911253-6737-45a9-93b5-2b1fc737889d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df2cd6c23ca843a4a7a89dbc9e835157', 'request_id': 'req-1b911253-6737-45a9-93b5-2b1fc737889d'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:31:19.820100 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-1b911253-6737-45a9-93b5-2b1fc737889d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df2cd6c23ca843a4a7a89dbc9e835157', 'request_id': 'req-1b911253-6737-45a9-93b5-2b1fc737889d'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:31:19.838870 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-1b911253-6737-45a9-93b5-2b1fc737889d admin admin] The token cache is being invalidate because project df2cd6c23ca843a4a7a89dbc9e835157 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:31:19.839353 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-1b911253-6737-45a9-93b5-2b1fc737889d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:31:19.840899 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-1b911253-6737-45a9-93b5-2b1fc737889d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:31:19.844827 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1156/2338] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:31:19 2022] DELETE /identity/v3/projects/df2cd6c23ca843a4a7a89dbc9e835157 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:31:19.850060 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90195d48-f83a-4597-ae5b-5b8c89227027 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:19.850604 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90195d48-f83a-4597-ae5b-5b8c89227027 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:19.850604 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90195d48-f83a-4597-ae5b-5b8c89227027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:19.872299 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-90195d48-f83a-4597-ae5b-5b8c89227027 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:31:19.873638 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-90195d48-f83a-4597-ae5b-5b8c89227027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:31:19.949178 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1183/2339] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:31:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:31:19.951359 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902ad0ef-96f4-4204-96e1-a0a999b8ef7a None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:19.951723 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902ad0ef-96f4-4204-96e1-a0a999b8ef7a None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:19.952010 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902ad0ef-96f4-4204-96e1-a0a999b8ef7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:19.975568 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-902ad0ef-96f4-4204-96e1-a0a999b8ef7a None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:31:19.976786 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-902ad0ef-96f4-4204-96e1-a0a999b8ef7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:31:19.986253 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1157/2340] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:31:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:31:19.991632 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90195d48-f83a-4597-ae5b-5b8c89227027 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:31:19.997410 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:31:20.006035 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90195d48-f83a-4597-ae5b-5b8c89227027 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:31:20.012768 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6587611f-22d6-472e-bb45-fb96bfd59f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:31:20.012768 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6587611f-22d6-472e-bb45-fb96bfd59f80 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:20.012768 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6587611f-22d6-472e-bb45-fb96bfd59f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:20.012768 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6587611f-22d6-472e-bb45-fb96bfd59f80 admin admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:20.013541 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6587611f-22d6-472e-bb45-fb96bfd59f80 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:31:20.013949 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6587611f-22d6-472e-bb45-fb96bfd59f80 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:31:20.015639 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6587611f-22d6-472e-bb45-fb96bfd59f80 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:31:20.033084 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1184/2341] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Aug 4 21:31:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:31:20.034015 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902ad0ef-96f4-4204-96e1-a0a999b8ef7a None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:31:20.044340 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902ad0ef-96f4-4204-96e1-a0a999b8ef7a None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:31:20.050182 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cb7186-ee33-481d-8d26-e2fb7d86b716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:31:20.051082 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cb7186-ee33-481d-8d26-e2fb7d86b716 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:20.051505 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cb7186-ee33-481d-8d26-e2fb7d86b716 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:20.051968 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cb7186-ee33-481d-8d26-e2fb7d86b716 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:20.052839 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cb7186-ee33-481d-8d26-e2fb7d86b716 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:31:20.055237 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cb7186-ee33-481d-8d26-e2fb7d86b716 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:31:20.185079 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1158/2342] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:31:20 2022] POST /identity/v3/projects => generated 361 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:31:20.189936 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6587611f-22d6-472e-bb45-fb96bfd59f80 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:31:20.214952 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6587611f-22d6-472e-bb45-fb96bfd59f80 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:31:20.284078 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e886ff83-9e34-477e-9094-a7de9a1609c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:31:20.284700 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e886ff83-9e34-477e-9094-a7de9a1609c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:20.284986 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e886ff83-9e34-477e-9094-a7de9a1609c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:20.285264 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e886ff83-9e34-477e-9094-a7de9a1609c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:20.286029 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e886ff83-9e34-477e-9094-a7de9a1609c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:31:20.287905 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e886ff83-9e34-477e-9094-a7de9a1609c2 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:31:20.403900 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1185/2343] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:31:20 2022] POST /identity/v3/users => generated 281 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:31:20.416686 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cb7186-ee33-481d-8d26-e2fb7d86b716 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:31:20.427944 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cb7186-ee33-481d-8d26-e2fb7d86b716 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:31:20.432515 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49857a8d-4a36-4f7d-86d8-c2a73df5d8b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:31:20.433124 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49857a8d-4a36-4f7d-86d8-c2a73df5d8b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:20.433401 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49857a8d-4a36-4f7d-86d8-c2a73df5d8b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:20.433670 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49857a8d-4a36-4f7d-86d8-c2a73df5d8b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:20.434354 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49857a8d-4a36-4f7d-86d8-c2a73df5d8b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:31:20.436088 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49857a8d-4a36-4f7d-86d8-c2a73df5d8b4 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:31:20.446853 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1159/2344] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:31:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:31:20.451001 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e886ff83-9e34-477e-9094-a7de9a1609c2 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:31:20.461814 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e886ff83-9e34-477e-9094-a7de9a1609c2 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:31:20.466807 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5220ff7-c22f-4188-a8b2-ccc68d36965d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:31:20.467553 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5220ff7-c22f-4188-a8b2-ccc68d36965d admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:20.467860 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5220ff7-c22f-4188-a8b2-ccc68d36965d admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:20.468141 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5220ff7-c22f-4188-a8b2-ccc68d36965d admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:20.468905 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5220ff7-c22f-4188-a8b2-ccc68d36965d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:31:20.471127 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5220ff7-c22f-4188-a8b2-ccc68d36965d admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:31:20.483858 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1186/2345] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:31:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:31:20.488776 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49857a8d-4a36-4f7d-86d8-c2a73df5d8b4 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:31:20.499603 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49857a8d-4a36-4f7d-86d8-c2a73df5d8b4 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:31:20.508943 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a86a42e-69d3-4654-a456-d94c9701da77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:31:20.509717 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a86a42e-69d3-4654-a456-d94c9701da77 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:20.510064 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a86a42e-69d3-4654-a456-d94c9701da77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:20.510425 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a86a42e-69d3-4654-a456-d94c9701da77 admin admin] PATH_INFO: `/v3/projects/d4962a8afc7e4d06bf119d331b392356/users/8e6a0428725e46af8ba41c59823e31d3/roles/8feebc60363c4feb870718b1721e5324` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:20.527111 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a86a42e-69d3-4654-a456-d94c9701da77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4962a8afc7e4d06bf119d331b392356, user_id=8e6a0428725e46af8ba41c59823e31d3, role_id=8feebc60363c4feb870718b1721e5324)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:31:20.529926 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a86a42e-69d3-4654-a456-d94c9701da77 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:31:20.634508 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1160/2346] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:31:20 2022] PUT /identity/v3/projects/d4962a8afc7e4d06bf119d331b392356/users/8e6a0428725e46af8ba41c59823e31d3/roles/8feebc60363c4feb870718b1721e5324 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:31:20.645496 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdafa55-4c65-4899-8b51-de0ab09dcb87 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:20.645794 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdafa55-4c65-4899-8b51-de0ab09dcb87 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:20.646057 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdafa55-4c65-4899-8b51-de0ab09dcb87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:20.675277 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-abdafa55-4c65-4899-8b51-de0ab09dcb87 None None] MFA Rules not processed for user `8e6a0428725e46af8ba41c59823e31d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:31:20.676710 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-abdafa55-4c65-4899-8b51-de0ab09dcb87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:31:20.825973 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1187/2347] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:31:20 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:31:20.865681 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a86a42e-69d3-4654-a456-d94c9701da77 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:31:20.867690 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-7a86a42e-69d3-4654-a456-d94c9701da77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:31:20.947956 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a86a42e-69d3-4654-a456-d94c9701da77 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:31:20.960796 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1bedef-7df5-4b7c-bc63-8c047a97dfa5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e31f2728056b460fa474478eb85ca1d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:31:20.961447 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1bedef-7df5-4b7c-bc63-8c047a97dfa5 service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:20.961735 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1bedef-7df5-4b7c-bc63-8c047a97dfa5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:20.962014 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1bedef-7df5-4b7c-bc63-8c047a97dfa5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:20.972703 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1bedef-7df5-4b7c-bc63-8c047a97dfa5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:31:20.974263 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1bedef-7df5-4b7c-bc63-8c047a97dfa5 service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:31:21.012560 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdafa55-4c65-4899-8b51-de0ab09dcb87 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:31:21.015668 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1161/2348] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:31:20 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:31:21.131046 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:31:21.136798 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdafa55-4c65-4899-8b51-de0ab09dcb87 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:31:21.143635 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbd2ebe-c1ce-4543-b97a-3dc3b04b178c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:31:21.144903 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbd2ebe-c1ce-4543-b97a-3dc3b04b178c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:21.145248 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbd2ebe-c1ce-4543-b97a-3dc3b04b178c admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:21.145536 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbd2ebe-c1ce-4543-b97a-3dc3b04b178c admin admin] PATH_INFO: `/v3/users/8e6a0428725e46af8ba41c59823e31d3` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:21.146881 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbd2ebe-c1ce-4543-b97a-3dc3b04b178c admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e6a0428725e46af8ba41c59823e31d3)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:31:21.149011 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbd2ebe-c1ce-4543-b97a-3dc3b04b178c admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:31:21.398052 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-adbd2ebe-c1ce-4543-b97a-3dc3b04b178c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e6a0428725e46af8ba41c59823e31d3'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:31:21.411524 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-adbd2ebe-c1ce-4543-b97a-3dc3b04b178c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e6a0428725e46af8ba41c59823e31d3'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:31:21.412428 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-adbd2ebe-c1ce-4543-b97a-3dc3b04b178c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e6a0428725e46af8ba41c59823e31d3'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:31:21.423069 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-adbd2ebe-c1ce-4543-b97a-3dc3b04b178c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e6a0428725e46af8ba41c59823e31d3'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:31:21.479188 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-adbd2ebe-c1ce-4543-b97a-3dc3b04b178c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e6a0428725e46af8ba41c59823e31d3'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:31:21.511800 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dfa4a4-a550-417f-b3b9-0d398644b998 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:21.512288 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dfa4a4-a550-417f-b3b9-0d398644b998 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:21.512562 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dfa4a4-a550-417f-b3b9-0d398644b998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:21.519380 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1188/2349] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:31:21 2022] DELETE /identity/v3/users/8e6a0428725e46af8ba41c59823e31d3 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:31:21.532491 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-34dfa4a4-a550-417f-b3b9-0d398644b998 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:31:21.534020 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-34dfa4a4-a550-417f-b3b9-0d398644b998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:31:21.575869 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:31:21.623011 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1162/2350] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:31:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:31:21.633109 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbd2ebe-c1ce-4543-b97a-3dc3b04b178c admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:31:21.634652 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-adbd2ebe-c1ce-4543-b97a-3dc3b04b178c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:31:21.710249 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbd2ebe-c1ce-4543-b97a-3dc3b04b178c admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:31:21.724603 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e23791-302f-4c90-bd5b-d7b4f0dbb665 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:31:21.725310 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e23791-302f-4c90-bd5b-d7b4f0dbb665 service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:21.725597 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e23791-302f-4c90-bd5b-d7b4f0dbb665 service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:21.725875 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e23791-302f-4c90-bd5b-d7b4f0dbb665 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:21.737652 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e23791-302f-4c90-bd5b-d7b4f0dbb665 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:31:21.739034 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e23791-302f-4c90-bd5b-d7b4f0dbb665 service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:31:21.750970 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1189/2351] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:31:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:31:22.551704 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dfa4a4-a550-417f-b3b9-0d398644b998 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:31:22.553634 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-34dfa4a4-a550-417f-b3b9-0d398644b998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:31:22.581954 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dfa4a4-a550-417f-b3b9-0d398644b998 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:31:22.589552 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b22713-9a39-4485-9abb-15e15a9754bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:31:22.590233 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b22713-9a39-4485-9abb-15e15a9754bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:22.590515 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b22713-9a39-4485-9abb-15e15a9754bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:22.590791 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b22713-9a39-4485-9abb-15e15a9754bf admin admin] PATH_INFO: `/v3/projects/d4962a8afc7e4d06bf119d331b392356` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:22.593965 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b22713-9a39-4485-9abb-15e15a9754bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4962a8afc7e4d06bf119d331b392356)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:31:22.598163 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b22713-9a39-4485-9abb-15e15a9754bf admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:31:22.743295 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-16b22713-9a39-4485-9abb-15e15a9754bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4962a8afc7e4d06bf119d331b392356', 'request_id': 'req-16b22713-9a39-4485-9abb-15e15a9754bf'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:31:22.753841 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-16b22713-9a39-4485-9abb-15e15a9754bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4962a8afc7e4d06bf119d331b392356', 'request_id': 'req-16b22713-9a39-4485-9abb-15e15a9754bf'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:31:22.847633 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-16b22713-9a39-4485-9abb-15e15a9754bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4962a8afc7e4d06bf119d331b392356', 'request_id': 'req-16b22713-9a39-4485-9abb-15e15a9754bf'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:31:22.864431 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-16b22713-9a39-4485-9abb-15e15a9754bf admin admin] The token cache is being invalidate because project d4962a8afc7e4d06bf119d331b392356 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:31:22.865723 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-16b22713-9a39-4485-9abb-15e15a9754bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:31:22.866590 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-16b22713-9a39-4485-9abb-15e15a9754bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:31:22.868412 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1163/2352] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:31:22 2022] DELETE /identity/v3/projects/d4962a8afc7e4d06bf119d331b392356 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:31:22.874148 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4268f059-846e-49c5-a818-35d13a165f34 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:22.874722 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4268f059-846e-49c5-a818-35d13a165f34 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:22.874722 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4268f059-846e-49c5-a818-35d13a165f34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:22.897918 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-4268f059-846e-49c5-a818-35d13a165f34 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:31:22.899142 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-4268f059-846e-49c5-a818-35d13a165f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:31:22.967101 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1190/2353] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:31:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:31:22.974528 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a9fc69-bc61-4b66-b6a5-691f2148a90f None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:22.974826 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a9fc69-bc61-4b66-b6a5-691f2148a90f None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:22.975080 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a9fc69-bc61-4b66-b6a5-691f2148a90f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:22.996077 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-f8a9fc69-bc61-4b66-b6a5-691f2148a90f None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:31:22.997355 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-f8a9fc69-bc61-4b66-b6a5-691f2148a90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:31:23.006804 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1164/2354] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:31:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:31:23.011228 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4268f059-846e-49c5-a818-35d13a165f34 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:31:23.033123 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4268f059-846e-49c5-a818-35d13a165f34 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:31:23.041194 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2f37cf-8c33-4068-a238-a078c029aa00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:31:23.042174 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2f37cf-8c33-4068-a238-a078c029aa00 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:23.042702 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2f37cf-8c33-4068-a238-a078c029aa00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:23.043234 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2f37cf-8c33-4068-a238-a078c029aa00 admin admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:23.044899 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2f37cf-8c33-4068-a238-a078c029aa00 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:31:23.045557 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2f37cf-8c33-4068-a238-a078c029aa00 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:31:23.049256 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2f37cf-8c33-4068-a238-a078c029aa00 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:31:23.061736 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1191/2355] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Aug 4 21:31:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:31:23.066845 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a9fc69-bc61-4b66-b6a5-691f2148a90f None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:31:23.084507 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a9fc69-bc61-4b66-b6a5-691f2148a90f None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:31:23.090226 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab51618-326d-42bb-a944-479eb0947f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:31:23.091597 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab51618-326d-42bb-a944-479eb0947f05 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:23.092225 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab51618-326d-42bb-a944-479eb0947f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:23.092824 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab51618-326d-42bb-a944-479eb0947f05 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:23.094183 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab51618-326d-42bb-a944-479eb0947f05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:31:23.098063 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab51618-326d-42bb-a944-479eb0947f05 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:31:23.253725 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1165/2356] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:31:23 2022] POST /identity/v3/projects => generated 375 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:31:23.258042 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2f37cf-8c33-4068-a238-a078c029aa00 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:31:23.268982 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2f37cf-8c33-4068-a238-a078c029aa00 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:31:23.274043 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:31:23.339843 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c907d82e-06f5-46fd-b875-f950c1a4878b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:31:23.339843 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c907d82e-06f5-46fd-b875-f950c1a4878b admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:23.339843 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c907d82e-06f5-46fd-b875-f950c1a4878b admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:23.339843 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c907d82e-06f5-46fd-b875-f950c1a4878b admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:23.340618 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c907d82e-06f5-46fd-b875-f950c1a4878b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:31:23.341850 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c907d82e-06f5-46fd-b875-f950c1a4878b admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:31:23.434076 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1192/2357] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:31:23 2022] POST /identity/v3/users => generated 288 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:31:23.438114 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab51618-326d-42bb-a944-479eb0947f05 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:31:23.451723 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab51618-326d-42bb-a944-479eb0947f05 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:31:23.470706 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af6413e-a44f-43d0-a182-b8ea6650511a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:31:23.471464 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af6413e-a44f-43d0-a182-b8ea6650511a admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:23.471752 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af6413e-a44f-43d0-a182-b8ea6650511a admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:23.472024 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af6413e-a44f-43d0-a182-b8ea6650511a admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:23.472751 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af6413e-a44f-43d0-a182-b8ea6650511a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:31:23.475272 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af6413e-a44f-43d0-a182-b8ea6650511a admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:31:23.490833 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1166/2358] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:31:23 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:31:23.492983 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c907d82e-06f5-46fd-b875-f950c1a4878b admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:31:23.504728 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c907d82e-06f5-46fd-b875-f950c1a4878b admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:31:23.509974 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de104de4-bb85-4bb4-b128-98b4e4a25661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:31:23.514772 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de104de4-bb85-4bb4-b128-98b4e4a25661 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:23.514772 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de104de4-bb85-4bb4-b128-98b4e4a25661 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:23.514772 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de104de4-bb85-4bb4-b128-98b4e4a25661 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:23.514772 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de104de4-bb85-4bb4-b128-98b4e4a25661 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:31:23.514772 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de104de4-bb85-4bb4-b128-98b4e4a25661 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:31:23.537735 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1193/2359] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:31:23 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:31:23.538870 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af6413e-a44f-43d0-a182-b8ea6650511a admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:31:23.550250 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af6413e-a44f-43d0-a182-b8ea6650511a admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:31:23.555804 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c70fb6-90cc-4a4a-a7d2-c5e1a7db513c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:31:23.557421 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c70fb6-90cc-4a4a-a7d2-c5e1a7db513c admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:23.557953 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c70fb6-90cc-4a4a-a7d2-c5e1a7db513c admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:23.558400 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c70fb6-90cc-4a4a-a7d2-c5e1a7db513c admin admin] PATH_INFO: `/v3/projects/b06f7b7fa4c4495baa349073f2d783e6/users/ff5669e1e521401c9b39e5d54c8a0d33/roles/8feebc60363c4feb870718b1721e5324` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:23.599556 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c70fb6-90cc-4a4a-a7d2-c5e1a7db513c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b06f7b7fa4c4495baa349073f2d783e6, user_id=ff5669e1e521401c9b39e5d54c8a0d33, role_id=8feebc60363c4feb870718b1721e5324)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:31:23.602489 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c70fb6-90cc-4a4a-a7d2-c5e1a7db513c admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:31:23.702938 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1167/2360] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:31:23 2022] PUT /identity/v3/projects/b06f7b7fa4c4495baa349073f2d783e6/users/ff5669e1e521401c9b39e5d54c8a0d33/roles/8feebc60363c4feb870718b1721e5324 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:31:23.714155 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9286e9d-be87-4ad3-a348-d8bfa1bafbd1 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:23.716657 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9286e9d-be87-4ad3-a348-d8bfa1bafbd1 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:23.716657 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9286e9d-be87-4ad3-a348-d8bfa1bafbd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:23.757238 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-d9286e9d-be87-4ad3-a348-d8bfa1bafbd1 None None] MFA Rules not processed for user `ff5669e1e521401c9b39e5d54c8a0d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:31:23.759033 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-d9286e9d-be87-4ad3-a348-d8bfa1bafbd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:31:23.860995 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1194/2361] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:31:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:31:23.872017 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c70fb6-90cc-4a4a-a7d2-c5e1a7db513c admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:31:23.873707 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-a7c70fb6-90cc-4a4a-a7d2-c5e1a7db513c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:31:23.916882 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:31:23.943345 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c70fb6-90cc-4a4a-a7d2-c5e1a7db513c admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:31:23.948406 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65080f6c-ff75-443b-924a-b6c1040101e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e31f2728056b460fa474478eb85ca1d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:31:23.949396 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65080f6c-ff75-443b-924a-b6c1040101e6 service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:23.949897 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65080f6c-ff75-443b-924a-b6c1040101e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:23.950284 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65080f6c-ff75-443b-924a-b6c1040101e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:23.959265 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65080f6c-ff75-443b-924a-b6c1040101e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:31:23.963250 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65080f6c-ff75-443b-924a-b6c1040101e6 service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:31:23.970600 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1168/2362] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:31:23 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:31:29.310294 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ec3cb4-e7c6-4cd1-86b5-57b0ae63bec1 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:29.310806 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ec3cb4-e7c6-4cd1-86b5-57b0ae63bec1 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:29.310909 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ec3cb4-e7c6-4cd1-86b5-57b0ae63bec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:29.340908 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-b0ec3cb4-e7c6-4cd1-86b5-57b0ae63bec1 None None] MFA Rules not processed for user `23ea380385f649628cc27e3603bc700c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:31:29.342044 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-b0ec3cb4-e7c6-4cd1-86b5-57b0ae63bec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:31:29.379638 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:31:29.419854 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1195/2363] 10.222.0.25 () {58 vars in 967 bytes} [Thu Aug 4 21:31:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:31:29.432627 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65080f6c-ff75-443b-924a-b6c1040101e6 service cinder] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:31:29.437644 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-65080f6c-ff75-443b-924a-b6c1040101e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:31:29.534067 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65080f6c-ff75-443b-924a-b6c1040101e6 service cinder] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:31:29.541939 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2fd1a7-4942-4399-bbca-93645b2b2c29 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58935367c0ad445f8477cdac0df04d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:31:29.542907 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2fd1a7-4942-4399-bbca-93645b2b2c29 service swift] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:29.542907 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2fd1a7-4942-4399-bbca-93645b2b2c29 service swift] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:29.543114 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2fd1a7-4942-4399-bbca-93645b2b2c29 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:29.553208 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2fd1a7-4942-4399-bbca-93645b2b2c29 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:31:29.554821 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2fd1a7-4942-4399-bbca-93645b2b2c29 service swift] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:31:29.563854 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1169/2364] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Aug 4 21:31:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 21:31:35.798840 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ec3cb4-e7c6-4cd1-86b5-57b0ae63bec1 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:31:35.882394 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ec3cb4-e7c6-4cd1-86b5-57b0ae63bec1 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:31:35.887140 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d722c4-85c5-4ed3-9e28-8b44eb90a7d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:31:35.887832 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d722c4-85c5-4ed3-9e28-8b44eb90a7d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:35.888112 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d722c4-85c5-4ed3-9e28-8b44eb90a7d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:35.888384 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d722c4-85c5-4ed3-9e28-8b44eb90a7d1 admin admin] PATH_INFO: `/v3/users/ff5669e1e521401c9b39e5d54c8a0d33` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:35.889610 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d722c4-85c5-4ed3-9e28-8b44eb90a7d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff5669e1e521401c9b39e5d54c8a0d33)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:31:35.892077 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d722c4-85c5-4ed3-9e28-8b44eb90a7d1 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:31:36.080008 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-02d722c4-85c5-4ed3-9e28-8b44eb90a7d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff5669e1e521401c9b39e5d54c8a0d33'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:31:36.090661 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-02d722c4-85c5-4ed3-9e28-8b44eb90a7d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff5669e1e521401c9b39e5d54c8a0d33'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:31:36.091470 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-02d722c4-85c5-4ed3-9e28-8b44eb90a7d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff5669e1e521401c9b39e5d54c8a0d33'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:31:36.092092 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-02d722c4-85c5-4ed3-9e28-8b44eb90a7d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff5669e1e521401c9b39e5d54c8a0d33'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:31:36.193236 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-02d722c4-85c5-4ed3-9e28-8b44eb90a7d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff5669e1e521401c9b39e5d54c8a0d33'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:31:36.206992 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1196/2365] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:31:35 2022] DELETE /identity/v3/users/ff5669e1e521401c9b39e5d54c8a0d33 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:31:36.211515 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5e4573-c364-4f2f-b3f2-b878737b52fd None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:36.212006 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5e4573-c364-4f2f-b3f2-b878737b52fd None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:36.212123 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5e4573-c364-4f2f-b3f2-b878737b52fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:36.241964 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-8c5e4573-c364-4f2f-b3f2-b878737b52fd None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:31:36.243231 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-8c5e4573-c364-4f2f-b3f2-b878737b52fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:31:36.326328 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1170/2366] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:31:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:31:36.336420 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d722c4-85c5-4ed3-9e28-8b44eb90a7d1 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:31:36.338069 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-02d722c4-85c5-4ed3-9e28-8b44eb90a7d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:31:36.373486 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:31:36.417504 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d722c4-85c5-4ed3-9e28-8b44eb90a7d1 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:31:36.421295 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0347f05-3972-43a8-89e1-bd69a44bf658 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:31:36.421915 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0347f05-3972-43a8-89e1-bd69a44bf658 service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:36.422194 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0347f05-3972-43a8-89e1-bd69a44bf658 service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:36.423072 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0347f05-3972-43a8-89e1-bd69a44bf658 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:36.432185 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0347f05-3972-43a8-89e1-bd69a44bf658 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:31:36.433610 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0347f05-3972-43a8-89e1-bd69a44bf658 service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:31:36.445044 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1197/2367] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:31:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:31:36.894941 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5e4573-c364-4f2f-b3f2-b878737b52fd None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:31:36.896661 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-8c5e4573-c364-4f2f-b3f2-b878737b52fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:31:36.913964 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5e4573-c364-4f2f-b3f2-b878737b52fd None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:31:36.919255 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e8ddb5-4fbe-458d-8c8a-0bb038596f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:31:36.919989 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e8ddb5-4fbe-458d-8c8a-0bb038596f28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:36.925645 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e8ddb5-4fbe-458d-8c8a-0bb038596f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:36.925645 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e8ddb5-4fbe-458d-8c8a-0bb038596f28 admin admin] PATH_INFO: `/v3/projects/b06f7b7fa4c4495baa349073f2d783e6` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:36.925645 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e8ddb5-4fbe-458d-8c8a-0bb038596f28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b06f7b7fa4c4495baa349073f2d783e6)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:31:36.925645 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e8ddb5-4fbe-458d-8c8a-0bb038596f28 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:31:37.048656 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:31:37.056896 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-07e8ddb5-4fbe-458d-8c8a-0bb038596f28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b06f7b7fa4c4495baa349073f2d783e6', 'request_id': 'req-07e8ddb5-4fbe-458d-8c8a-0bb038596f28'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:31:37.062355 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-07e8ddb5-4fbe-458d-8c8a-0bb038596f28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b06f7b7fa4c4495baa349073f2d783e6', 'request_id': 'req-07e8ddb5-4fbe-458d-8c8a-0bb038596f28'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:31:37.130487 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-07e8ddb5-4fbe-458d-8c8a-0bb038596f28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b06f7b7fa4c4495baa349073f2d783e6', 'request_id': 'req-07e8ddb5-4fbe-458d-8c8a-0bb038596f28'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:31:37.142935 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-07e8ddb5-4fbe-458d-8c8a-0bb038596f28 admin admin] The token cache is being invalidate because project b06f7b7fa4c4495baa349073f2d783e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:31:37.143162 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-07e8ddb5-4fbe-458d-8c8a-0bb038596f28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:31:37.143845 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-07e8ddb5-4fbe-458d-8c8a-0bb038596f28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:31:37.145440 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1171/2368] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:31:36 2022] DELETE /identity/v3/projects/b06f7b7fa4c4495baa349073f2d783e6 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:31:37.150799 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af264584-dc12-4de9-b1cb-bbc7ac69a302 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:37.151270 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af264584-dc12-4de9-b1cb-bbc7ac69a302 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:37.151410 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af264584-dc12-4de9-b1cb-bbc7ac69a302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:37.210372 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-af264584-dc12-4de9-b1cb-bbc7ac69a302 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:31:37.211616 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-af264584-dc12-4de9-b1cb-bbc7ac69a302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:31:37.272538 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1198/2369] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:31:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:31:37.280354 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b780aaa-d2be-4094-b6ab-8a388531d078 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:37.280856 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b780aaa-d2be-4094-b6ab-8a388531d078 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:37.280955 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b780aaa-d2be-4094-b6ab-8a388531d078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:37.300790 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-5b780aaa-d2be-4094-b6ab-8a388531d078 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:31:37.301992 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-5b780aaa-d2be-4094-b6ab-8a388531d078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:31:37.309863 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1172/2370] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:31:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:31:37.314209 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af264584-dc12-4de9-b1cb-bbc7ac69a302 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:31:37.324334 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af264584-dc12-4de9-b1cb-bbc7ac69a302 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:31:37.329412 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1a7213-c024-4436-83ae-ba9bb8529024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:31:37.330014 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1a7213-c024-4436-83ae-ba9bb8529024 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:37.330295 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1a7213-c024-4436-83ae-ba9bb8529024 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:37.330566 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1a7213-c024-4436-83ae-ba9bb8529024 admin admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:37.331264 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1a7213-c024-4436-83ae-ba9bb8529024 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:31:37.331600 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1a7213-c024-4436-83ae-ba9bb8529024 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:31:37.333197 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1a7213-c024-4436-83ae-ba9bb8529024 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:31:37.345889 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1199/2371] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Aug 4 21:31:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:31:37.350307 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b780aaa-d2be-4094-b6ab-8a388531d078 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:31:37.359734 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b780aaa-d2be-4094-b6ab-8a388531d078 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:31:37.364473 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4996dd-da7d-41d6-af93-cbf9c408229c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:31:37.365086 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4996dd-da7d-41d6-af93-cbf9c408229c admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:37.365371 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4996dd-da7d-41d6-af93-cbf9c408229c admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:37.365646 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4996dd-da7d-41d6-af93-cbf9c408229c admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:37.366304 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4996dd-da7d-41d6-af93-cbf9c408229c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:31:37.368278 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4996dd-da7d-41d6-af93-cbf9c408229c admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:31:37.486193 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1173/2372] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:31:37 2022] POST /identity/v3/projects => generated 373 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:31:37.490599 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1a7213-c024-4436-83ae-ba9bb8529024 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:31:37.501897 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1a7213-c024-4436-83ae-ba9bb8529024 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:31:37.559098 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf34041-73b3-4814-85bd-fe7a70d316ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:31:37.559756 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf34041-73b3-4814-85bd-fe7a70d316ec admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:37.560038 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf34041-73b3-4814-85bd-fe7a70d316ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:37.560314 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf34041-73b3-4814-85bd-fe7a70d316ec admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:37.560972 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf34041-73b3-4814-85bd-fe7a70d316ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:31:37.563444 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf34041-73b3-4814-85bd-fe7a70d316ec admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:31:37.637870 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1200/2373] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:31:37 2022] POST /identity/v3/users => generated 287 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:31:37.642764 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4996dd-da7d-41d6-af93-cbf9c408229c admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:31:37.653738 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4996dd-da7d-41d6-af93-cbf9c408229c admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:31:37.658858 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808d30e8-6f12-4682-b9cc-330918a38b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:31:37.659510 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808d30e8-6f12-4682-b9cc-330918a38b54 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:37.659798 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808d30e8-6f12-4682-b9cc-330918a38b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:37.660075 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808d30e8-6f12-4682-b9cc-330918a38b54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:37.660782 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808d30e8-6f12-4682-b9cc-330918a38b54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:31:37.662476 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808d30e8-6f12-4682-b9cc-330918a38b54 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:31:37.672943 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1174/2374] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:31:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:31:37.677122 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf34041-73b3-4814-85bd-fe7a70d316ec admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:31:37.686975 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf34041-73b3-4814-85bd-fe7a70d316ec admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:31:37.692374 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece0651d-c9b8-4d48-9681-97feed8e15ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:31:37.692991 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece0651d-c9b8-4d48-9681-97feed8e15ab admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:37.693278 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece0651d-c9b8-4d48-9681-97feed8e15ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:37.693553 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece0651d-c9b8-4d48-9681-97feed8e15ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:37.694250 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece0651d-c9b8-4d48-9681-97feed8e15ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:31:37.696155 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece0651d-c9b8-4d48-9681-97feed8e15ab admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:31:37.697576 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:31:37.710759 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1201/2375] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:31:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:31:37.712929 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808d30e8-6f12-4682-b9cc-330918a38b54 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:31:37.723790 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808d30e8-6f12-4682-b9cc-330918a38b54 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:31:37.728908 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21185242-062e-4e6b-9b6a-147ef02c19e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:31:37.729585 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21185242-062e-4e6b-9b6a-147ef02c19e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:37.729872 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21185242-062e-4e6b-9b6a-147ef02c19e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:37.730948 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21185242-062e-4e6b-9b6a-147ef02c19e9 admin admin] PATH_INFO: `/v3/projects/5603eb09b4be40b98fe3bad6c40680b7/users/f6a9a76ad8914f28927418359e5a5eee/roles/8feebc60363c4feb870718b1721e5324` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:37.745583 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21185242-062e-4e6b-9b6a-147ef02c19e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5603eb09b4be40b98fe3bad6c40680b7, user_id=f6a9a76ad8914f28927418359e5a5eee, role_id=8feebc60363c4feb870718b1721e5324)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:31:37.748361 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21185242-062e-4e6b-9b6a-147ef02c19e9 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:31:37.815117 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1175/2376] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:31:37 2022] PUT /identity/v3/projects/5603eb09b4be40b98fe3bad6c40680b7/users/f6a9a76ad8914f28927418359e5a5eee/roles/8feebc60363c4feb870718b1721e5324 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:31:37.826780 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8edba7-8b53-4f2f-9d03-d18c4bb91488 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:37.827138 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8edba7-8b53-4f2f-9d03-d18c4bb91488 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:37.827524 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8edba7-8b53-4f2f-9d03-d18c4bb91488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:37.856466 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-2a8edba7-8b53-4f2f-9d03-d18c4bb91488 None None] MFA Rules not processed for user `f6a9a76ad8914f28927418359e5a5eee`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:31:37.857712 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-2a8edba7-8b53-4f2f-9d03-d18c4bb91488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:31:37.924064 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1202/2377] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:31:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:31:37.940441 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21185242-062e-4e6b-9b6a-147ef02c19e9 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:31:37.942145 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-21185242-062e-4e6b-9b6a-147ef02c19e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:31:38.033347 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21185242-062e-4e6b-9b6a-147ef02c19e9 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:31:38.036285 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1e3539-e0e5-49e8-af5c-89999526dc80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e31f2728056b460fa474478eb85ca1d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:31:38.036627 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1e3539-e0e5-49e8-af5c-89999526dc80 service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:38.036923 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1e3539-e0e5-49e8-af5c-89999526dc80 service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:38.037187 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1e3539-e0e5-49e8-af5c-89999526dc80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:38.046288 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1e3539-e0e5-49e8-af5c-89999526dc80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:31:38.048315 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1e3539-e0e5-49e8-af5c-89999526dc80 service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:31:38.057736 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1176/2378] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:31:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:31:39.842406 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9904fc6-b227-4ec8-b35c-3ac724e2e67c None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:39.842922 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9904fc6-b227-4ec8-b35c-3ac724e2e67c None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:39.843018 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9904fc6-b227-4ec8-b35c-3ac724e2e67c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:39.863414 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-c9904fc6-b227-4ec8-b35c-3ac724e2e67c None None] MFA Rules not processed for user `f6a9a76ad8914f28927418359e5a5eee`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:31:39.864629 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-c9904fc6-b227-4ec8-b35c-3ac724e2e67c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:31:39.871841 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1203/2379] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:31:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:31:48.053508 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=77,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:31:49.858552 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=37 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:31:55.340685 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1e3539-e0e5-49e8-af5c-89999526dc80 service cinder] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:31:55.342197 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-ee1e3539-e0e5-49e8-af5c-89999526dc80 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:31:55.401385 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1e3539-e0e5-49e8-af5c-89999526dc80 service cinder] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:31:55.405371 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcaf7e1-0519-4383-b70e-55a51d60a04a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:31:55.406064 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcaf7e1-0519-4383-b70e-55a51d60a04a service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:55.406359 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcaf7e1-0519-4383-b70e-55a51d60a04a service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:55.406698 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcaf7e1-0519-4383-b70e-55a51d60a04a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:55.408572 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-1fcaf7e1-0519-4383-b70e-55a51d60a04a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:31:55.490342 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcaf7e1-0519-4383-b70e-55a51d60a04a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:31:55.492011 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcaf7e1-0519-4383-b70e-55a51d60a04a service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:31:55.501939 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1177/2380] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:31:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:31:56.521649 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9904fc6-b227-4ec8-b35c-3ac724e2e67c None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:31:56.534924 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9904fc6-b227-4ec8-b35c-3ac724e2e67c None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:31:56.538794 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2768c77f-f636-421c-8d07-95cd837677cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:31:56.539534 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2768c77f-f636-421c-8d07-95cd837677cd service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:31:56.539819 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2768c77f-f636-421c-8d07-95cd837677cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:31:56.540097 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2768c77f-f636-421c-8d07-95cd837677cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:31:56.542073 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-2768c77f-f636-421c-8d07-95cd837677cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:31:56.556254 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2768c77f-f636-421c-8d07-95cd837677cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:31:56.557945 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2768c77f-f636-421c-8d07-95cd837677cd service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:31:56.571392 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1204/2381] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:31:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:32:00.752909 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6307ca-1aa4-46f2-8f31-f0f7955f6f68 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:00.756736 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6307ca-1aa4-46f2-8f31-f0f7955f6f68 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:00.756736 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6307ca-1aa4-46f2-8f31-f0f7955f6f68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:00.775584 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-ea6307ca-1aa4-46f2-8f31-f0f7955f6f68 None None] MFA Rules not processed for user `23ea380385f649628cc27e3603bc700c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:32:00.780012 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-ea6307ca-1aa4-46f2-8f31-f0f7955f6f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:00.841557 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1178/2382] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Aug 4 21:32:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:32:00.867135 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2768c77f-f636-421c-8d07-95cd837677cd service neutron] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:00.868633 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-2768c77f-f636-421c-8d07-95cd837677cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:00.954946 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2768c77f-f636-421c-8d07-95cd837677cd service neutron] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:00.962482 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4094c24a-4341-4ada-b970-817061874067 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58935367c0ad445f8477cdac0df04d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:00.963134 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4094c24a-4341-4ada-b970-817061874067 service swift] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:00.968416 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4094c24a-4341-4ada-b970-817061874067 service swift] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:00.969264 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4094c24a-4341-4ada-b970-817061874067 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:00.982593 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4094c24a-4341-4ada-b970-817061874067 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:00.984267 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4094c24a-4341-4ada-b970-817061874067 service swift] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:00.995290 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1205/2383] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Aug 4 21:32:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 21:32:01.859785 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6307ca-1aa4-46f2-8f31-f0f7955f6f68 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:01.862064 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-ea6307ca-1aa4-46f2-8f31-f0f7955f6f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:01.966917 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:01.983493 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6307ca-1aa4-46f2-8f31-f0f7955f6f68 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:01.991424 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d75f8ae-190c-4085-9e80-0fb332a0b42e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67d44a0098ab4c36b14b36e9e59ae75c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:01.992129 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d75f8ae-190c-4085-9e80-0fb332a0b42e service placement] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:01.992541 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d75f8ae-190c-4085-9e80-0fb332a0b42e service placement] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:01.992915 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d75f8ae-190c-4085-9e80-0fb332a0b42e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:01.995258 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-6d75f8ae-190c-4085-9e80-0fb332a0b42e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:02.008527 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d75f8ae-190c-4085-9e80-0fb332a0b42e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:02.010287 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d75f8ae-190c-4085-9e80-0fb332a0b42e service placement] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:02.026447 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1179/2384] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:32:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:32:02.083940 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4094c24a-4341-4ada-b970-817061874067 service swift] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:02.086643 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-4094c24a-4341-4ada-b970-817061874067 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:02.104635 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4094c24a-4341-4ada-b970-817061874067 service swift] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:02.109630 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b1b6c6-ecaf-4e6f-ba0f-bc6298a69b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:02.110452 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b1b6c6-ecaf-4e6f-ba0f-bc6298a69b73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:02.111026 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b1b6c6-ecaf-4e6f-ba0f-bc6298a69b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:02.111210 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b1b6c6-ecaf-4e6f-ba0f-bc6298a69b73 admin admin] PATH_INFO: `/v3/users/9b29c3f2c29d4e2aba552c144cff94f8` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:02.112609 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b1b6c6-ecaf-4e6f-ba0f-bc6298a69b73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b29c3f2c29d4e2aba552c144cff94f8)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:02.115575 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b1b6c6-ecaf-4e6f-ba0f-bc6298a69b73 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:02.224871 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ec7953-c2ba-4e53-aae9-265df4b5b2ff None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:02.225487 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ec7953-c2ba-4e53-aae9-265df4b5b2ff None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:02.225861 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ec7953-c2ba-4e53-aae9-265df4b5b2ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:02.249000 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-a7ec7953-c2ba-4e53-aae9-265df4b5b2ff None None] MFA Rules not processed for user `23ea380385f649628cc27e3603bc700c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:32:02.250821 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-a7ec7953-c2ba-4e53-aae9-265df4b5b2ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:02.259964 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1180/2385] 10.222.0.25 () {58 vars in 967 bytes} [Thu Aug 4 21:32:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:32:02.272433 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-20b1b6c6-ecaf-4e6f-ba0f-bc6298a69b73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b29c3f2c29d4e2aba552c144cff94f8'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:02.273765 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ec7953-c2ba-4e53-aae9-265df4b5b2ff None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:02.290708 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-20b1b6c6-ecaf-4e6f-ba0f-bc6298a69b73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b29c3f2c29d4e2aba552c144cff94f8'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:02.292161 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ec7953-c2ba-4e53-aae9-265df4b5b2ff None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:02.293398 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-20b1b6c6-ecaf-4e6f-ba0f-bc6298a69b73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b29c3f2c29d4e2aba552c144cff94f8'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:02.294318 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-20b1b6c6-ecaf-4e6f-ba0f-bc6298a69b73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b29c3f2c29d4e2aba552c144cff94f8'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:02.299764 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7dd2dd-5fac-4f2d-8119-c37fe8592c22 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58935367c0ad445f8477cdac0df04d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:02.300499 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7dd2dd-5fac-4f2d-8119-c37fe8592c22 service swift] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:02.300966 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7dd2dd-5fac-4f2d-8119-c37fe8592c22 service swift] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:02.301353 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7dd2dd-5fac-4f2d-8119-c37fe8592c22 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:02.303303 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-7d7dd2dd-5fac-4f2d-8119-c37fe8592c22 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:02.320634 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7dd2dd-5fac-4f2d-8119-c37fe8592c22 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:02.322274 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7dd2dd-5fac-4f2d-8119-c37fe8592c22 service swift] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:02.331335 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1181/2386] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Aug 4 21:32:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 04 21:32:02.367743 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-20b1b6c6-ecaf-4e6f-ba0f-bc6298a69b73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b29c3f2c29d4e2aba552c144cff94f8'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:02.397632 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1206/2387] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:32:02 2022] DELETE /identity/v3/users/9b29c3f2c29d4e2aba552c144cff94f8 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:32:02.512064 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7dd2dd-5fac-4f2d-8119-c37fe8592c22 service swift] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:02.513483 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-7d7dd2dd-5fac-4f2d-8119-c37fe8592c22 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:02.529353 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7dd2dd-5fac-4f2d-8119-c37fe8592c22 service swift] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:02.595732 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd837c49-d37c-43b7-9efa-3e75d0983172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:02.596383 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd837c49-d37c-43b7-9efa-3e75d0983172 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:02.596669 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd837c49-d37c-43b7-9efa-3e75d0983172 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:02.596945 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd837c49-d37c-43b7-9efa-3e75d0983172 admin admin] PATH_INFO: `/v3/users/ea2f8480bc7d44f6a51e4a1d875a1c7c` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:02.598300 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd837c49-d37c-43b7-9efa-3e75d0983172 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea2f8480bc7d44f6a51e4a1d875a1c7c)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:02.600195 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd837c49-d37c-43b7-9efa-3e75d0983172 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:02.682903 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b1b6c6-ecaf-4e6f-ba0f-bc6298a69b73 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:02.684363 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-20b1b6c6-ecaf-4e6f-ba0f-bc6298a69b73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:02.701093 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b1b6c6-ecaf-4e6f-ba0f-bc6298a69b73 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:02.708232 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f765f77-657b-4748-b37d-2fcf1dac37fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:02.708993 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f765f77-657b-4748-b37d-2fcf1dac37fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:02.709235 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f765f77-657b-4748-b37d-2fcf1dac37fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:02.709516 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f765f77-657b-4748-b37d-2fcf1dac37fb admin admin] PATH_INFO: `/v3/projects/00814c2455be4608b2984adcf6353df2` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:02.710724 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f765f77-657b-4748-b37d-2fcf1dac37fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=00814c2455be4608b2984adcf6353df2)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:02.713733 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f765f77-657b-4748-b37d-2fcf1dac37fb admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:02.846571 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-fd837c49-d37c-43b7-9efa-3e75d0983172 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea2f8480bc7d44f6a51e4a1d875a1c7c'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:02.858229 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-fd837c49-d37c-43b7-9efa-3e75d0983172 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea2f8480bc7d44f6a51e4a1d875a1c7c'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:02.859101 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-fd837c49-d37c-43b7-9efa-3e75d0983172 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea2f8480bc7d44f6a51e4a1d875a1c7c'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:02.866238 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-fd837c49-d37c-43b7-9efa-3e75d0983172 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea2f8480bc7d44f6a51e4a1d875a1c7c'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:02.870122 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-fd837c49-d37c-43b7-9efa-3e75d0983172 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea2f8480bc7d44f6a51e4a1d875a1c7c'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:02.876437 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:02.880088 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-0f765f77-657b-4748-b37d-2fcf1dac37fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00814c2455be4608b2984adcf6353df2', 'request_id': 'req-0f765f77-657b-4748-b37d-2fcf1dac37fb'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:02.885462 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-0f765f77-657b-4748-b37d-2fcf1dac37fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00814c2455be4608b2984adcf6353df2', 'request_id': 'req-0f765f77-657b-4748-b37d-2fcf1dac37fb'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:03.014657 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1182/2388] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:32:02 2022] DELETE /identity/v3/users/ea2f8480bc7d44f6a51e4a1d875a1c7c => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:32:03.091032 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cbc3bf-9373-484c-aeaa-c1f697b8a3f0 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:03.091440 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cbc3bf-9373-484c-aeaa-c1f697b8a3f0 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:03.091791 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cbc3bf-9373-484c-aeaa-c1f697b8a3f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:03.102078 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-0f765f77-657b-4748-b37d-2fcf1dac37fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00814c2455be4608b2984adcf6353df2', 'request_id': 'req-0f765f77-657b-4748-b37d-2fcf1dac37fb'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:03.115528 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-21cbc3bf-9373-484c-aeaa-c1f697b8a3f0 None None] MFA Rules not processed for user `23ea380385f649628cc27e3603bc700c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:32:03.117404 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-21cbc3bf-9373-484c-aeaa-c1f697b8a3f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:03.132817 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-0f765f77-657b-4748-b37d-2fcf1dac37fb admin admin] The token cache is being invalidate because project 00814c2455be4608b2984adcf6353df2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:32:03.133350 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-0f765f77-657b-4748-b37d-2fcf1dac37fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:03.134230 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-0f765f77-657b-4748-b37d-2fcf1dac37fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:03.140484 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1207/2389] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:32:02 2022] DELETE /identity/v3/projects/00814c2455be4608b2984adcf6353df2 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:32:03.147781 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b0a239-a1df-42e3-ad8b-886c1b290809 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:03.147781 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b0a239-a1df-42e3-ad8b-886c1b290809 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:03.147781 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b0a239-a1df-42e3-ad8b-886c1b290809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:03.167575 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-c9b0a239-a1df-42e3-ad8b-886c1b290809 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:32:03.168802 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-c9b0a239-a1df-42e3-ad8b-886c1b290809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:03.170730 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:03.187263 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1183/2390] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Aug 4 21:32:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:32:03.202829 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cbc3bf-9373-484c-aeaa-c1f697b8a3f0 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:03.204927 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-21cbc3bf-9373-484c-aeaa-c1f697b8a3f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:03.253191 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1208/2391] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:32:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:32:03.261170 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5b212e-a6fc-4490-8d5e-592d09cbcf89 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:03.261459 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5b212e-a6fc-4490-8d5e-592d09cbcf89 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:03.261738 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5b212e-a6fc-4490-8d5e-592d09cbcf89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:03.281678 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-7b5b212e-a6fc-4490-8d5e-592d09cbcf89 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:32:03.282872 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-7b5b212e-a6fc-4490-8d5e-592d09cbcf89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:03.287499 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cbc3bf-9373-484c-aeaa-c1f697b8a3f0 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:03.290884 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1209/2392] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:32:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:32:03.292744 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d3c9bd-2b58-408e-95f0-15e1e24bcea8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58935367c0ad445f8477cdac0df04d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:03.293369 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d3c9bd-2b58-408e-95f0-15e1e24bcea8 service swift] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:03.293648 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d3c9bd-2b58-408e-95f0-15e1e24bcea8 service swift] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:03.294103 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d3c9bd-2b58-408e-95f0-15e1e24bcea8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:03.294977 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5b212e-a6fc-4490-8d5e-592d09cbcf89 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:03.303248 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d3c9bd-2b58-408e-95f0-15e1e24bcea8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:03.304632 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5b212e-a6fc-4490-8d5e-592d09cbcf89 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:03.304752 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d3c9bd-2b58-408e-95f0-15e1e24bcea8 service swift] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:03.309300 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f168c418-4965-488e-ae89-7c3fd799de30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:03.309900 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f168c418-4965-488e-ae89-7c3fd799de30 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:03.310320 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f168c418-4965-488e-ae89-7c3fd799de30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:03.310601 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f168c418-4965-488e-ae89-7c3fd799de30 admin admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:03.311288 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f168c418-4965-488e-ae89-7c3fd799de30 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:32:03.316196 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f168c418-4965-488e-ae89-7c3fd799de30 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:03.317259 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1184/2393] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Aug 4 21:32:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 04 21:32:03.317610 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f168c418-4965-488e-ae89-7c3fd799de30 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:03.328500 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1210/2394] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Aug 4 21:32:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:32:03.332973 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d3c9bd-2b58-408e-95f0-15e1e24bcea8 service swift] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:03.343101 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d3c9bd-2b58-408e-95f0-15e1e24bcea8 service swift] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:03.347828 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd2bb87-2956-41a1-a1ee-ed87a3d76b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:03.348532 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd2bb87-2956-41a1-a1ee-ed87a3d76b66 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:03.348747 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd2bb87-2956-41a1-a1ee-ed87a3d76b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:03.349021 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd2bb87-2956-41a1-a1ee-ed87a3d76b66 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:03.349720 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd2bb87-2956-41a1-a1ee-ed87a3d76b66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:03.351668 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd2bb87-2956-41a1-a1ee-ed87a3d76b66 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:03.371470 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f168c418-4965-488e-ae89-7c3fd799de30 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:03.373237 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-f168c418-4965-488e-ae89-7c3fd799de30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:03.388215 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f168c418-4965-488e-ae89-7c3fd799de30 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:03.394598 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b63cf8-965f-4f29-ae25-2762b5b18abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:03.395252 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b63cf8-965f-4f29-ae25-2762b5b18abd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:03.395599 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b63cf8-965f-4f29-ae25-2762b5b18abd admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:03.395874 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b63cf8-965f-4f29-ae25-2762b5b18abd admin admin] PATH_INFO: `/v3/projects/f28e5935db5c4be4871d727e1958424b` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:03.397234 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b63cf8-965f-4f29-ae25-2762b5b18abd admin admin] RBAC: Authorizing `identity:delete_project(project_id=f28e5935db5c4be4871d727e1958424b)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:03.399125 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b63cf8-965f-4f29-ae25-2762b5b18abd admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:03.479795 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1185/2395] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:32:03 2022] POST /identity/v3/projects => generated 385 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:32:03.483973 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd2bb87-2956-41a1-a1ee-ed87a3d76b66 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:03.554610 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd2bb87-2956-41a1-a1ee-ed87a3d76b66 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:03.556852 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-81b63cf8-965f-4f29-ae25-2762b5b18abd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f28e5935db5c4be4871d727e1958424b', 'request_id': 'req-81b63cf8-965f-4f29-ae25-2762b5b18abd'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:03.563884 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-81b63cf8-965f-4f29-ae25-2762b5b18abd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f28e5935db5c4be4871d727e1958424b', 'request_id': 'req-81b63cf8-965f-4f29-ae25-2762b5b18abd'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:03.615772 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab134c4-dafd-466c-96e5-80edd2bc03ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:03.616399 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab134c4-dafd-466c-96e5-80edd2bc03ef admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:03.616678 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab134c4-dafd-466c-96e5-80edd2bc03ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:03.617123 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab134c4-dafd-466c-96e5-80edd2bc03ef admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:03.617764 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab134c4-dafd-466c-96e5-80edd2bc03ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:03.620586 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab134c4-dafd-466c-96e5-80edd2bc03ef admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:03.631307 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-81b63cf8-965f-4f29-ae25-2762b5b18abd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f28e5935db5c4be4871d727e1958424b', 'request_id': 'req-81b63cf8-965f-4f29-ae25-2762b5b18abd'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:03.643399 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-81b63cf8-965f-4f29-ae25-2762b5b18abd admin admin] The token cache is being invalidate because project f28e5935db5c4be4871d727e1958424b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:32:03.643691 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-81b63cf8-965f-4f29-ae25-2762b5b18abd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:03.644518 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-81b63cf8-965f-4f29-ae25-2762b5b18abd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:03.646225 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1211/2396] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:32:03 2022] DELETE /identity/v3/projects/f28e5935db5c4be4871d727e1958424b => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:32:03.651007 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7368df86-45e1-4e0c-999c-12734d5ae538 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:03.652221 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7368df86-45e1-4e0c-999c-12734d5ae538 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:03.652221 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7368df86-45e1-4e0c-999c-12734d5ae538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:03.670864 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-7368df86-45e1-4e0c-999c-12734d5ae538 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:32:03.672680 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-7368df86-45e1-4e0c-999c-12734d5ae538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:03.681057 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1212/2397] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:32:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:32:03.690263 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d15a6b-b62f-475f-baf7-b8c3ffe96cee None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:03.690547 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d15a6b-b62f-475f-baf7-b8c3ffe96cee None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:03.690814 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d15a6b-b62f-475f-baf7-b8c3ffe96cee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:03.705836 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1186/2398] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:32:03 2022] POST /identity/v3/users => generated 293 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:32:03.709973 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab134c4-dafd-466c-96e5-80edd2bc03ef admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:03.711913 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-eab134c4-dafd-466c-96e5-80edd2bc03ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:03.713477 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-82d15a6b-b62f-475f-baf7-b8c3ffe96cee None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:32:03.714650 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-82d15a6b-b62f-475f-baf7-b8c3ffe96cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:03.726426 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab134c4-dafd-466c-96e5-80edd2bc03ef admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:03.727733 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1213/2399] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:32:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:32:03.730399 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d15a6b-b62f-475f-baf7-b8c3ffe96cee None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:03.734341 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25307c47-ecaa-4043-964b-99bb71b92c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:03.735865 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25307c47-ecaa-4043-964b-99bb71b92c9e admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:03.735990 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25307c47-ecaa-4043-964b-99bb71b92c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:03.736182 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25307c47-ecaa-4043-964b-99bb71b92c9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:03.736893 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25307c47-ecaa-4043-964b-99bb71b92c9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:03.738942 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25307c47-ecaa-4043-964b-99bb71b92c9e admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:03.740831 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d15a6b-b62f-475f-baf7-b8c3ffe96cee None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:03.745802 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f5e553-3931-42de-843c-057cbca49384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:03.746447 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f5e553-3931-42de-843c-057cbca49384 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:03.746731 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f5e553-3931-42de-843c-057cbca49384 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:03.747031 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f5e553-3931-42de-843c-057cbca49384 admin admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:03.747815 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f5e553-3931-42de-843c-057cbca49384 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:32:03.748144 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f5e553-3931-42de-843c-057cbca49384 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:03.749630 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1187/2400] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:32:03 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:32:03.749774 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f5e553-3931-42de-843c-057cbca49384 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:03.754486 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25307c47-ecaa-4043-964b-99bb71b92c9e admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:03.762291 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1214/2401] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Aug 4 21:32:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:32:03.768614 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25307c47-ecaa-4043-964b-99bb71b92c9e admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:03.770988 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f5e553-3931-42de-843c-057cbca49384 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:03.773654 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fc4724-8202-41da-a952-52b893764bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:03.774353 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fc4724-8202-41da-a952-52b893764bac admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:03.774709 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fc4724-8202-41da-a952-52b893764bac admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:03.775054 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fc4724-8202-41da-a952-52b893764bac admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:03.775851 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fc4724-8202-41da-a952-52b893764bac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:03.777544 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fc4724-8202-41da-a952-52b893764bac admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:03.784227 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f5e553-3931-42de-843c-057cbca49384 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:03.790501 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7013e53-0811-4680-a714-eb65d0e59d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:03.791122 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7013e53-0811-4680-a714-eb65d0e59d5f admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:03.791525 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7013e53-0811-4680-a714-eb65d0e59d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:03.798034 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7013e53-0811-4680-a714-eb65d0e59d5f admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:03.798269 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1188/2402] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:32:03 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:32:03.798368 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7013e53-0811-4680-a714-eb65d0e59d5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:03.798368 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7013e53-0811-4680-a714-eb65d0e59d5f admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:03.801419 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fc4724-8202-41da-a952-52b893764bac admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:03.803398 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:03.813461 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fc4724-8202-41da-a952-52b893764bac admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:03.818657 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af313bcb-c1e7-4f64-a12a-2e7fc969652e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:03.819352 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af313bcb-c1e7-4f64-a12a-2e7fc969652e admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:03.819644 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af313bcb-c1e7-4f64-a12a-2e7fc969652e admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:03.820491 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af313bcb-c1e7-4f64-a12a-2e7fc969652e admin admin] PATH_INFO: `/v3/projects/7e578ef042424a7d9b74a085efaa7d4b/users/1b7bda63d2764ffdb325bae10afa5ea5/roles/8feebc60363c4feb870718b1721e5324` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:03.837619 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af313bcb-c1e7-4f64-a12a-2e7fc969652e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e578ef042424a7d9b74a085efaa7d4b, user_id=1b7bda63d2764ffdb325bae10afa5ea5, role_id=8feebc60363c4feb870718b1721e5324)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:03.840401 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af313bcb-c1e7-4f64-a12a-2e7fc969652e admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:03.886692 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1215/2403] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:32:03 2022] POST /identity/v3/projects => generated 373 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:32:03.887731 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7013e53-0811-4680-a714-eb65d0e59d5f admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:03.902149 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1189/2404] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:32:03 2022] PUT /identity/v3/projects/7e578ef042424a7d9b74a085efaa7d4b/users/1b7bda63d2764ffdb325bae10afa5ea5/roles/8feebc60363c4feb870718b1721e5324 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:32:03.920145 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5058c0e1-dda1-4b70-953c-9d51cbb6f065 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:03.920564 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5058c0e1-dda1-4b70-953c-9d51cbb6f065 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:03.920845 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5058c0e1-dda1-4b70-953c-9d51cbb6f065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:03.952546 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-5058c0e1-dda1-4b70-953c-9d51cbb6f065 None None] MFA Rules not processed for user `1b7bda63d2764ffdb325bae10afa5ea5`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:32:03.953766 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-5058c0e1-dda1-4b70-953c-9d51cbb6f065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:03.958174 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7013e53-0811-4680-a714-eb65d0e59d5f admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:04.022231 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1c86fc-2bbd-40da-a0dc-50cc0cc3439a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:04.022958 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1c86fc-2bbd-40da-a0dc-50cc0cc3439a admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:04.023299 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1c86fc-2bbd-40da-a0dc-50cc0cc3439a admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:04.023720 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1c86fc-2bbd-40da-a0dc-50cc0cc3439a admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:04.029550 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1c86fc-2bbd-40da-a0dc-50cc0cc3439a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:04.029677 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1190/2405] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:32:03 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:32:04.029677 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed8b6b0-37f3-4af1-8141-fd393ecb154b None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:04.029677 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed8b6b0-37f3-4af1-8141-fd393ecb154b None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:04.029677 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed8b6b0-37f3-4af1-8141-fd393ecb154b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:04.030027 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1c86fc-2bbd-40da-a0dc-50cc0cc3439a admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:04.052766 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-7ed8b6b0-37f3-4af1-8141-fd393ecb154b None None] MFA Rules not processed for user `23ea380385f649628cc27e3603bc700c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:32:04.052766 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-7ed8b6b0-37f3-4af1-8141-fd393ecb154b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:04.109872 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1191/2406] 10.222.0.25 () {58 vars in 967 bytes} [Thu Aug 4 21:32:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:32:04.111808 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed8b6b0-37f3-4af1-8141-fd393ecb154b None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:04.123038 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed8b6b0-37f3-4af1-8141-fd393ecb154b None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:04.128236 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e87b93-b57e-4dde-9670-2f88d951064b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:04.132938 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e87b93-b57e-4dde-9670-2f88d951064b admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:04.132938 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e87b93-b57e-4dde-9670-2f88d951064b admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:04.132938 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e87b93-b57e-4dde-9670-2f88d951064b admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:04.132938 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e87b93-b57e-4dde-9670-2f88d951064b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:04.133298 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1216/2407] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:32:03 2022] POST /identity/v3/users => generated 287 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:32:04.133298 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1c86fc-2bbd-40da-a0dc-50cc0cc3439a admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:04.133827 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e87b93-b57e-4dde-9670-2f88d951064b admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:04.147727 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-3d1c86fc-2bbd-40da-a0dc-50cc0cc3439a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:04.230634 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1c86fc-2bbd-40da-a0dc-50cc0cc3439a admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:04.236232 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efda93b9-4de4-4ff3-819d-39d92c7c4c06 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58935367c0ad445f8477cdac0df04d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:04.237304 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda93b9-4de4-4ff3-819d-39d92c7c4c06 service swift] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:04.237304 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda93b9-4de4-4ff3-819d-39d92c7c4c06 service swift] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:04.237548 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda93b9-4de4-4ff3-819d-39d92c7c4c06 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:04.247184 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efda93b9-4de4-4ff3-819d-39d92c7c4c06 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:04.249043 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efda93b9-4de4-4ff3-819d-39d92c7c4c06 service swift] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:04.257466 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1192/2408] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:32:04 2022] POST /identity/v3/projects => generated 387 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:32:04.258969 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e87b93-b57e-4dde-9670-2f88d951064b admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:04.261664 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1217/2409] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Aug 4 21:32:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 21:32:04.262531 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efda93b9-4de4-4ff3-819d-39d92c7c4c06 service swift] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:04.276152 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efda93b9-4de4-4ff3-819d-39d92c7c4c06 service swift] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:04.279282 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:04.341934 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7deaaf-0e0f-4941-b641-e6883cfd3665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:04.342560 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7deaaf-0e0f-4941-b641-e6883cfd3665 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:04.342845 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7deaaf-0e0f-4941-b641-e6883cfd3665 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:04.343115 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7deaaf-0e0f-4941-b641-e6883cfd3665 admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:04.343814 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7deaaf-0e0f-4941-b641-e6883cfd3665 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:04.346591 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7deaaf-0e0f-4941-b641-e6883cfd3665 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:04.350110 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e87b93-b57e-4dde-9670-2f88d951064b admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:04.354168 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd84a250-ef93-4906-9064-09af3c76ba20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:04.354794 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd84a250-ef93-4906-9064-09af3c76ba20 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:04.355084 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd84a250-ef93-4906-9064-09af3c76ba20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:04.355389 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd84a250-ef93-4906-9064-09af3c76ba20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:04.356097 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd84a250-ef93-4906-9064-09af3c76ba20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:04.357714 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd84a250-ef93-4906-9064-09af3c76ba20 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:04.367229 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:04.369991 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1193/2410] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:32:04 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:32:04.372377 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd84a250-ef93-4906-9064-09af3c76ba20 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:04.382317 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd84a250-ef93-4906-9064-09af3c76ba20 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:04.387298 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce26040e-41a3-424c-be41-46bbe5f1c044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:04.387946 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce26040e-41a3-424c-be41-46bbe5f1c044 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:04.388286 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce26040e-41a3-424c-be41-46bbe5f1c044 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:04.388511 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce26040e-41a3-424c-be41-46bbe5f1c044 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:04.389208 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce26040e-41a3-424c-be41-46bbe5f1c044 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:04.391527 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce26040e-41a3-424c-be41-46bbe5f1c044 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:04.401401 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1194/2411] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:32:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:32:04.405795 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce26040e-41a3-424c-be41-46bbe5f1c044 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:04.415641 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce26040e-41a3-424c-be41-46bbe5f1c044 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:04.420218 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f9795f-c52e-4ee7-be45-c995299bcdbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:04.424811 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f9795f-c52e-4ee7-be45-c995299bcdbe admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:04.424811 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f9795f-c52e-4ee7-be45-c995299bcdbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:04.424811 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f9795f-c52e-4ee7-be45-c995299bcdbe admin admin] PATH_INFO: `/v3/projects/20b096f3bc2943e8b662a6745ac25a00/users/d99e04fa3d44457e8f88fe755d30ff7e/roles/8feebc60363c4feb870718b1721e5324` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:04.436241 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f9795f-c52e-4ee7-be45-c995299bcdbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=20b096f3bc2943e8b662a6745ac25a00, user_id=d99e04fa3d44457e8f88fe755d30ff7e, role_id=8feebc60363c4feb870718b1721e5324)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:04.439362 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f9795f-c52e-4ee7-be45-c995299bcdbe admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:04.450490 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1218/2412] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:32:04 2022] POST /identity/v3/users => generated 300 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:32:04.454920 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7deaaf-0e0f-4941-b641-e6883cfd3665 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:04.464841 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7deaaf-0e0f-4941-b641-e6883cfd3665 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:04.469832 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e89b398-ed25-45b0-a4e6-893152bced85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:04.470423 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e89b398-ed25-45b0-a4e6-893152bced85 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:04.470705 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e89b398-ed25-45b0-a4e6-893152bced85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:04.470974 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e89b398-ed25-45b0-a4e6-893152bced85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:04.471685 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e89b398-ed25-45b0-a4e6-893152bced85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:04.473539 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e89b398-ed25-45b0-a4e6-893152bced85 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:04.476477 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:04.487405 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1219/2413] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:32:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:32:04.492067 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e89b398-ed25-45b0-a4e6-893152bced85 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:04.503412 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e89b398-ed25-45b0-a4e6-893152bced85 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:04.508115 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a22a58-c8ba-4c52-95d7-8fb9254baf74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:04.508783 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a22a58-c8ba-4c52-95d7-8fb9254baf74 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:04.509064 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a22a58-c8ba-4c52-95d7-8fb9254baf74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:04.509339 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a22a58-c8ba-4c52-95d7-8fb9254baf74 admin admin] PATH_INFO: `/v3/projects/1fb7498643b84db4882816b5e1931b92/users/d10b1ef9971f4fcfb97d1ee2636fb979/roles/a432dd4735944ecfab2fa5ce1705d356` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:04.524873 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a22a58-c8ba-4c52-95d7-8fb9254baf74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fb7498643b84db4882816b5e1931b92, user_id=d10b1ef9971f4fcfb97d1ee2636fb979, role_id=a432dd4735944ecfab2fa5ce1705d356)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:04.528031 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a22a58-c8ba-4c52-95d7-8fb9254baf74 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:04.558138 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1195/2414] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:32:04 2022] PUT /identity/v3/projects/20b096f3bc2943e8b662a6745ac25a00/users/d99e04fa3d44457e8f88fe755d30ff7e/roles/8feebc60363c4feb870718b1721e5324 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:32:04.567390 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeae1dd-2e0b-420d-b3af-fe7a5610f65e None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:04.567681 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeae1dd-2e0b-420d-b3af-fe7a5610f65e None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:04.567959 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeae1dd-2e0b-420d-b3af-fe7a5610f65e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:04.597379 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-8eeae1dd-2e0b-420d-b3af-fe7a5610f65e None None] MFA Rules not processed for user `d99e04fa3d44457e8f88fe755d30ff7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:32:04.598727 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-8eeae1dd-2e0b-420d-b3af-fe7a5610f65e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:04.638111 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1220/2415] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:32:04 2022] PUT /identity/v3/projects/1fb7498643b84db4882816b5e1931b92/users/d10b1ef9971f4fcfb97d1ee2636fb979/roles/a432dd4735944ecfab2fa5ce1705d356 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:32:04.643054 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a22a58-c8ba-4c52-95d7-8fb9254baf74 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:04.672426 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1196/2416] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:32:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:32:04.676965 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeae1dd-2e0b-420d-b3af-fe7a5610f65e None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:04.727794 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a22a58-c8ba-4c52-95d7-8fb9254baf74 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:04.734980 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143cea62-bd65-497e-86fc-7c6fd89bb23a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:04.735886 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143cea62-bd65-497e-86fc-7c6fd89bb23a admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:04.736173 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143cea62-bd65-497e-86fc-7c6fd89bb23a admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:04.736520 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143cea62-bd65-497e-86fc-7c6fd89bb23a admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:04.737493 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143cea62-bd65-497e-86fc-7c6fd89bb23a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:04.739431 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143cea62-bd65-497e-86fc-7c6fd89bb23a admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:04.754629 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1221/2417] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:32:04 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:32:04.764721 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143cea62-bd65-497e-86fc-7c6fd89bb23a admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:04.778488 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143cea62-bd65-497e-86fc-7c6fd89bb23a admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:04.783559 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e173066-ffdc-4288-bea2-63e64134342c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:04.784372 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e173066-ffdc-4288-bea2-63e64134342c admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:04.791289 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e173066-ffdc-4288-bea2-63e64134342c admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:04.791289 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e173066-ffdc-4288-bea2-63e64134342c admin admin] PATH_INFO: `/v3/projects/1fb7498643b84db4882816b5e1931b92/users/d10b1ef9971f4fcfb97d1ee2636fb979/roles/a432dd4735944ecfab2fa5ce1705d356` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:04.791289 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e173066-ffdc-4288-bea2-63e64134342c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fb7498643b84db4882816b5e1931b92, user_id=d10b1ef9971f4fcfb97d1ee2636fb979, role_id=a432dd4735944ecfab2fa5ce1705d356)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:04.791289 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e173066-ffdc-4288-bea2-63e64134342c admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:04.798136 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeae1dd-2e0b-420d-b3af-fe7a5610f65e None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:04.799614 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b050b41-b1c4-4d00-9802-ae8f3c21e6fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:04.800347 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b050b41-b1c4-4d00-9802-ae8f3c21e6fc admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:04.800521 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b050b41-b1c4-4d00-9802-ae8f3c21e6fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:04.800792 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b050b41-b1c4-4d00-9802-ae8f3c21e6fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:04.802422 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b050b41-b1c4-4d00-9802-ae8f3c21e6fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:04.804404 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b050b41-b1c4-4d00-9802-ae8f3c21e6fc admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:04.807857 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1222/2418] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:32:04 2022] PUT /identity/v3/projects/1fb7498643b84db4882816b5e1931b92/users/d10b1ef9971f4fcfb97d1ee2636fb979/roles/a432dd4735944ecfab2fa5ce1705d356 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:32:04.818655 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aaab9f-e0ca-4b8f-83fa-d0ea4ac050d8 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:04.818943 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aaab9f-e0ca-4b8f-83fa-d0ea4ac050d8 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:04.819217 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aaab9f-e0ca-4b8f-83fa-d0ea4ac050d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:04.843384 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:04.856418 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-58aaab9f-e0ca-4b8f-83fa-d0ea4ac050d8 None None] MFA Rules not processed for user `d10b1ef9971f4fcfb97d1ee2636fb979`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:32:04.857631 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-58aaab9f-e0ca-4b8f-83fa-d0ea4ac050d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:04.906208 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:04.951622 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1223/2419] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:32:04 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:32:04.958731 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1197/2420] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:32:04 2022] POST /identity/v3/projects => generated 371 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:32:04.959251 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aaab9f-e0ca-4b8f-83fa-d0ea4ac050d8 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:04.976302 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b050b41-b1c4-4d00-9802-ae8f3c21e6fc admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:04.982624 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-4b050b41-b1c4-4d00-9802-ae8f3c21e6fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:05.067858 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aaab9f-e0ca-4b8f-83fa-d0ea4ac050d8 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:05.073256 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b050b41-b1c4-4d00-9802-ae8f3c21e6fc admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:05.078165 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6e68f5-716b-42ba-b0fa-6794e199fede service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e31f2728056b460fa474478eb85ca1d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:05.079208 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6e68f5-716b-42ba-b0fa-6794e199fede service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:05.079741 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6e68f5-716b-42ba-b0fa-6794e199fede service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:05.080231 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6e68f5-716b-42ba-b0fa-6794e199fede service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:05.160074 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5095a12c-c97b-4462-9c20-6b51379e2851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:05.160998 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5095a12c-c97b-4462-9c20-6b51379e2851 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:05.161410 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5095a12c-c97b-4462-9c20-6b51379e2851 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:05.161761 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5095a12c-c97b-4462-9c20-6b51379e2851 admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:05.162672 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5095a12c-c97b-4462-9c20-6b51379e2851 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:05.164984 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5095a12c-c97b-4462-9c20-6b51379e2851 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:05.199843 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6e68f5-716b-42ba-b0fa-6794e199fede service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:05.200838 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6e68f5-716b-42ba-b0fa-6794e199fede service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:05.211822 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1198/2421] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:32:04 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:32:05.251953 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6e68f5-716b-42ba-b0fa-6794e199fede service cinder] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:05.263199 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6e68f5-716b-42ba-b0fa-6794e199fede service cinder] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:05.267932 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb321cd2-94ac-438f-a8fa-55f18ea06393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:05.268571 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb321cd2-94ac-438f-a8fa-55f18ea06393 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:05.269004 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb321cd2-94ac-438f-a8fa-55f18ea06393 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:05.269308 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb321cd2-94ac-438f-a8fa-55f18ea06393 admin admin] PATH_INFO: `/v3/users/1b7bda63d2764ffdb325bae10afa5ea5` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:05.270531 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb321cd2-94ac-438f-a8fa-55f18ea06393 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b7bda63d2764ffdb325bae10afa5ea5)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:05.272535 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb321cd2-94ac-438f-a8fa-55f18ea06393 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:05.299172 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1224/2422] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:32:04 2022] POST /identity/v3/users => generated 292 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:32:05.306248 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5095a12c-c97b-4462-9c20-6b51379e2851 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:05.316430 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5095a12c-c97b-4462-9c20-6b51379e2851 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:05.321324 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257485ac-e96f-4590-b2e9-dd7fe758e88f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:05.321932 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257485ac-e96f-4590-b2e9-dd7fe758e88f admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:05.321932 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257485ac-e96f-4590-b2e9-dd7fe758e88f admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:05.321932 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257485ac-e96f-4590-b2e9-dd7fe758e88f admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:05.321932 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257485ac-e96f-4590-b2e9-dd7fe758e88f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:05.324958 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257485ac-e96f-4590-b2e9-dd7fe758e88f admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:05.338629 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1225/2423] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:32:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:32:05.343500 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257485ac-e96f-4590-b2e9-dd7fe758e88f admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:05.356039 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257485ac-e96f-4590-b2e9-dd7fe758e88f admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:05.361805 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989fa3cb-f06a-4cc2-b9e7-a997268ae692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:05.362482 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989fa3cb-f06a-4cc2-b9e7-a997268ae692 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:05.362791 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989fa3cb-f06a-4cc2-b9e7-a997268ae692 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:05.363138 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989fa3cb-f06a-4cc2-b9e7-a997268ae692 admin admin] PATH_INFO: `/v3/projects/b9b793168cc44baa9cce816ee861e197/users/f6d553a59f8c43d3b65513c4c7b58c70/roles/a432dd4735944ecfab2fa5ce1705d356` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:05.382154 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989fa3cb-f06a-4cc2-b9e7-a997268ae692 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9b793168cc44baa9cce816ee861e197, user_id=f6d553a59f8c43d3b65513c4c7b58c70, role_id=a432dd4735944ecfab2fa5ce1705d356)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:05.385456 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989fa3cb-f06a-4cc2-b9e7-a997268ae692 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:05.487592 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1226/2424] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:32:05 2022] PUT /identity/v3/projects/b9b793168cc44baa9cce816ee861e197/users/f6d553a59f8c43d3b65513c4c7b58c70/roles/a432dd4735944ecfab2fa5ce1705d356 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:32:05.492055 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989fa3cb-f06a-4cc2-b9e7-a997268ae692 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:05.540522 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-bb321cd2-94ac-438f-a8fa-55f18ea06393 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b7bda63d2764ffdb325bae10afa5ea5'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:05.557022 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-bb321cd2-94ac-438f-a8fa-55f18ea06393 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b7bda63d2764ffdb325bae10afa5ea5'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:05.557652 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-bb321cd2-94ac-438f-a8fa-55f18ea06393 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b7bda63d2764ffdb325bae10afa5ea5'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:05.558450 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:05.569038 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-bb321cd2-94ac-438f-a8fa-55f18ea06393 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b7bda63d2764ffdb325bae10afa5ea5'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:05.569853 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-bb321cd2-94ac-438f-a8fa-55f18ea06393 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b7bda63d2764ffdb325bae10afa5ea5'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:05.580683 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989fa3cb-f06a-4cc2-b9e7-a997268ae692 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:05.588593 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feac5b4-d580-4ba7-bf6a-07feff4a58e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:05.589336 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feac5b4-d580-4ba7-bf6a-07feff4a58e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:05.589689 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feac5b4-d580-4ba7-bf6a-07feff4a58e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:05.590035 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feac5b4-d580-4ba7-bf6a-07feff4a58e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:05.590811 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2feac5b4-d580-4ba7-bf6a-07feff4a58e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:05.592615 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2feac5b4-d580-4ba7-bf6a-07feff4a58e2 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:05.604629 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1227/2425] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:32:05 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:32:05.608812 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feac5b4-d580-4ba7-bf6a-07feff4a58e2 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:05.610835 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-2feac5b4-d580-4ba7-bf6a-07feff4a58e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:05.623808 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feac5b4-d580-4ba7-bf6a-07feff4a58e2 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:05.628944 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bfff9a-20bb-45a7-ba77-1203bda7d177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:05.629653 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bfff9a-20bb-45a7-ba77-1203bda7d177 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:05.629990 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bfff9a-20bb-45a7-ba77-1203bda7d177 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:05.630267 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bfff9a-20bb-45a7-ba77-1203bda7d177 admin admin] PATH_INFO: `/v3/projects/b9b793168cc44baa9cce816ee861e197/users/f6d553a59f8c43d3b65513c4c7b58c70/roles/a432dd4735944ecfab2fa5ce1705d356` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:05.632282 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bfff9a-20bb-45a7-ba77-1203bda7d177 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9b793168cc44baa9cce816ee861e197, user_id=f6d553a59f8c43d3b65513c4c7b58c70, role_id=a432dd4735944ecfab2fa5ce1705d356)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:05.635063 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bfff9a-20bb-45a7-ba77-1203bda7d177 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:05.648777 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1228/2426] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:32:05 2022] PUT /identity/v3/projects/b9b793168cc44baa9cce816ee861e197/users/f6d553a59f8c43d3b65513c4c7b58c70/roles/a432dd4735944ecfab2fa5ce1705d356 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:32:05.660740 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1199/2427] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:32:05 2022] DELETE /identity/v3/users/1b7bda63d2764ffdb325bae10afa5ea5 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:32:05.661124 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eef0594-8b91-4bee-9547-268b5e68b2e0 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:05.661353 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eef0594-8b91-4bee-9547-268b5e68b2e0 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:05.661627 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eef0594-8b91-4bee-9547-268b5e68b2e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:05.665935 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b57763-1058-4402-bc08-8470964d2fe3 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:05.666498 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b57763-1058-4402-bc08-8470964d2fe3 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:05.666844 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b57763-1058-4402-bc08-8470964d2fe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:05.702459 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-6eef0594-8b91-4bee-9547-268b5e68b2e0 None None] MFA Rules not processed for user `f6d553a59f8c43d3b65513c4c7b58c70`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:32:05.705430 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-6eef0594-8b91-4bee-9547-268b5e68b2e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:05.708610 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-a0b57763-1058-4402-bc08-8470964d2fe3 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:32:05.711266 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-a0b57763-1058-4402-bc08-8470964d2fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:05.741558 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:05.806294 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1229/2428] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:32:05 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:32:05.810644 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1200/2429] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:32:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:32:05.828249 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b57763-1058-4402-bc08-8470964d2fe3 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:05.829218 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eef0594-8b91-4bee-9547-268b5e68b2e0 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:05.831611 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-a0b57763-1058-4402-bc08-8470964d2fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:05.832208 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-6eef0594-8b91-4bee-9547-268b5e68b2e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:05.905896 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eef0594-8b91-4bee-9547-268b5e68b2e0 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:05.912704 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400206bc-6879-427b-b969-bfe3adcfbe28 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e31f2728056b460fa474478eb85ca1d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:05.913503 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400206bc-6879-427b-b969-bfe3adcfbe28 service cinder] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:05.913781 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400206bc-6879-427b-b969-bfe3adcfbe28 service cinder] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:05.914053 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400206bc-6879-427b-b969-bfe3adcfbe28 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:05.935709 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400206bc-6879-427b-b969-bfe3adcfbe28 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:05.936751 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400206bc-6879-427b-b969-bfe3adcfbe28 service cinder] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:05.948461 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b57763-1058-4402-bc08-8470964d2fe3 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:05.951017 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1230/2430] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:32:05 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:32:05.953283 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36f8473-8384-40eb-8754-9f90fcedc94f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:05.959191 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36f8473-8384-40eb-8754-9f90fcedc94f service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:05.959191 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36f8473-8384-40eb-8754-9f90fcedc94f service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:05.959191 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36f8473-8384-40eb-8754-9f90fcedc94f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:05.968294 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36f8473-8384-40eb-8754-9f90fcedc94f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:05.971002 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36f8473-8384-40eb-8754-9f90fcedc94f service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:05.984812 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1201/2431] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:32:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:32:06.485745 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400206bc-6879-427b-b969-bfe3adcfbe28 service cinder] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:06.487275 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-400206bc-6879-427b-b969-bfe3adcfbe28 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:06.502601 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400206bc-6879-427b-b969-bfe3adcfbe28 service cinder] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:06.507565 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687a79e9-597c-41a5-81cf-bd8b0d1430e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:06.508206 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687a79e9-597c-41a5-81cf-bd8b0d1430e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:06.508483 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687a79e9-597c-41a5-81cf-bd8b0d1430e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:06.508753 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687a79e9-597c-41a5-81cf-bd8b0d1430e8 admin admin] PATH_INFO: `/v3/projects/7e578ef042424a7d9b74a085efaa7d4b` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:06.510063 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687a79e9-597c-41a5-81cf-bd8b0d1430e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e578ef042424a7d9b74a085efaa7d4b)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:06.512216 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687a79e9-597c-41a5-81cf-bd8b0d1430e8 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:06.731258 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:06.732773 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-687a79e9-597c-41a5-81cf-bd8b0d1430e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e578ef042424a7d9b74a085efaa7d4b', 'request_id': 'req-687a79e9-597c-41a5-81cf-bd8b0d1430e8'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:06.744474 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-687a79e9-597c-41a5-81cf-bd8b0d1430e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e578ef042424a7d9b74a085efaa7d4b', 'request_id': 'req-687a79e9-597c-41a5-81cf-bd8b0d1430e8'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:06.841565 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-687a79e9-597c-41a5-81cf-bd8b0d1430e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e578ef042424a7d9b74a085efaa7d4b', 'request_id': 'req-687a79e9-597c-41a5-81cf-bd8b0d1430e8'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:06.854397 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-687a79e9-597c-41a5-81cf-bd8b0d1430e8 admin admin] The token cache is being invalidate because project 7e578ef042424a7d9b74a085efaa7d4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:32:06.854778 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-687a79e9-597c-41a5-81cf-bd8b0d1430e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:06.855540 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-687a79e9-597c-41a5-81cf-bd8b0d1430e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:06.857184 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1231/2432] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:32:06 2022] DELETE /identity/v3/projects/7e578ef042424a7d9b74a085efaa7d4b => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:32:06.861382 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36f8473-8384-40eb-8754-9f90fcedc94f service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:06.871497 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36f8473-8384-40eb-8754-9f90fcedc94f service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:06.941522 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad717eb-e5e5-46f1-8e8a-6b7a1523f594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:06.942275 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad717eb-e5e5-46f1-8e8a-6b7a1523f594 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:06.942626 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad717eb-e5e5-46f1-8e8a-6b7a1523f594 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:06.942979 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad717eb-e5e5-46f1-8e8a-6b7a1523f594 admin admin] PATH_INFO: `/v3/users/d10b1ef9971f4fcfb97d1ee2636fb979` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:06.944429 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad717eb-e5e5-46f1-8e8a-6b7a1523f594 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d10b1ef9971f4fcfb97d1ee2636fb979)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:06.946642 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad717eb-e5e5-46f1-8e8a-6b7a1523f594 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:07.230520 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687a79e9-597c-41a5-81cf-bd8b0d1430e8 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:07.236036 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-687a79e9-597c-41a5-81cf-bd8b0d1430e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:07.244292 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-cad717eb-e5e5-46f1-8e8a-6b7a1523f594 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd10b1ef9971f4fcfb97d1ee2636fb979'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:07.248589 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687a79e9-597c-41a5-81cf-bd8b0d1430e8 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:07.254227 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f06724-d0a7-46d9-a491-1c0851b13065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:07.255092 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f06724-d0a7-46d9-a491-1c0851b13065 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:07.255431 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f06724-d0a7-46d9-a491-1c0851b13065 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:07.255909 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f06724-d0a7-46d9-a491-1c0851b13065 admin admin] PATH_INFO: `/v3/users/d99e04fa3d44457e8f88fe755d30ff7e` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:07.257396 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f06724-d0a7-46d9-a491-1c0851b13065 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d99e04fa3d44457e8f88fe755d30ff7e)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:07.257958 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:07.259116 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-cad717eb-e5e5-46f1-8e8a-6b7a1523f594 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd10b1ef9971f4fcfb97d1ee2636fb979'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:07.259880 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-cad717eb-e5e5-46f1-8e8a-6b7a1523f594 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd10b1ef9971f4fcfb97d1ee2636fb979'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:07.260005 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f06724-d0a7-46d9-a491-1c0851b13065 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:07.267011 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-cad717eb-e5e5-46f1-8e8a-6b7a1523f594 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd10b1ef9971f4fcfb97d1ee2636fb979'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:07.267766 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-cad717eb-e5e5-46f1-8e8a-6b7a1523f594 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd10b1ef9971f4fcfb97d1ee2636fb979'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:07.347165 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1202/2433] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:32:06 2022] DELETE /identity/v3/users/d10b1ef9971f4fcfb97d1ee2636fb979 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:32:07.443851 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad717eb-e5e5-46f1-8e8a-6b7a1523f594 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:07.445597 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-cad717eb-e5e5-46f1-8e8a-6b7a1523f594 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:07.509370 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-42f06724-d0a7-46d9-a491-1c0851b13065 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd99e04fa3d44457e8f88fe755d30ff7e'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:07.509924 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad717eb-e5e5-46f1-8e8a-6b7a1523f594 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:07.514487 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6efae58-757c-42d3-97d2-6e5b1b51415a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:07.515122 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6efae58-757c-42d3-97d2-6e5b1b51415a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:07.515430 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6efae58-757c-42d3-97d2-6e5b1b51415a admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:07.515707 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6efae58-757c-42d3-97d2-6e5b1b51415a admin admin] PATH_INFO: `/v3/users/f6a9a76ad8914f28927418359e5a5eee` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:07.516909 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6efae58-757c-42d3-97d2-6e5b1b51415a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6a9a76ad8914f28927418359e5a5eee)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:07.518837 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6efae58-757c-42d3-97d2-6e5b1b51415a admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:07.521879 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-42f06724-d0a7-46d9-a491-1c0851b13065 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd99e04fa3d44457e8f88fe755d30ff7e'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:07.522599 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-42f06724-d0a7-46d9-a491-1c0851b13065 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd99e04fa3d44457e8f88fe755d30ff7e'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:07.523225 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-42f06724-d0a7-46d9-a491-1c0851b13065 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd99e04fa3d44457e8f88fe755d30ff7e'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:07.629955 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-42f06724-d0a7-46d9-a491-1c0851b13065 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd99e04fa3d44457e8f88fe755d30ff7e'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:07.646072 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1232/2434] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:32:07 2022] DELETE /identity/v3/users/d99e04fa3d44457e8f88fe755d30ff7e => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:32:07.649542 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6279c3f5-810d-45e6-9da1-b9bbc1b3066a None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:07.650083 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6279c3f5-810d-45e6-9da1-b9bbc1b3066a None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:07.650605 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6279c3f5-810d-45e6-9da1-b9bbc1b3066a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:07.674417 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-6279c3f5-810d-45e6-9da1-b9bbc1b3066a None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:32:07.676001 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-6279c3f5-810d-45e6-9da1-b9bbc1b3066a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:07.748646 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1233/2435] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:32:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:32:07.758357 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6279c3f5-810d-45e6-9da1-b9bbc1b3066a None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:07.759802 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-6279c3f5-810d-45e6-9da1-b9bbc1b3066a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:07.821078 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:07.839702 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6279c3f5-810d-45e6-9da1-b9bbc1b3066a None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:07.846232 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1c5dfb-c264-4d6c-806f-df82830db230 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:07.847057 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1c5dfb-c264-4d6c-806f-df82830db230 service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:07.847630 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1c5dfb-c264-4d6c-806f-df82830db230 service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:07.848066 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1c5dfb-c264-4d6c-806f-df82830db230 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:07.861094 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-f6efae58-757c-42d3-97d2-6e5b1b51415a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6a9a76ad8914f28927418359e5a5eee'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:07.862389 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1c5dfb-c264-4d6c-806f-df82830db230 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:07.864541 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1c5dfb-c264-4d6c-806f-df82830db230 service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:07.896908 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1234/2436] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:32:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:32:07.896908 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1c5dfb-c264-4d6c-806f-df82830db230 service neutron] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:07.896908 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-5d1c5dfb-c264-4d6c-806f-df82830db230 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:07.911350 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-f6efae58-757c-42d3-97d2-6e5b1b51415a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6a9a76ad8914f28927418359e5a5eee'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:07.914798 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-f6efae58-757c-42d3-97d2-6e5b1b51415a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6a9a76ad8914f28927418359e5a5eee'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:07.939003 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1c5dfb-c264-4d6c-806f-df82830db230 service neutron] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:07.948119 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-f6efae58-757c-42d3-97d2-6e5b1b51415a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6a9a76ad8914f28927418359e5a5eee'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:07.948940 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-f6efae58-757c-42d3-97d2-6e5b1b51415a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6a9a76ad8914f28927418359e5a5eee'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:07.949395 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeba4385-05e0-437d-8c2f-af77ed76477c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:07.950509 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeba4385-05e0-437d-8c2f-af77ed76477c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:07.950979 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeba4385-05e0-437d-8c2f-af77ed76477c admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:07.951367 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeba4385-05e0-437d-8c2f-af77ed76477c admin admin] PATH_INFO: `/v3/projects/1fb7498643b84db4882816b5e1931b92` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:07.954367 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeba4385-05e0-437d-8c2f-af77ed76477c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fb7498643b84db4882816b5e1931b92)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:07.956795 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeba4385-05e0-437d-8c2f-af77ed76477c admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:08.086365 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1203/2437] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:32:07 2022] DELETE /identity/v3/users/f6a9a76ad8914f28927418359e5a5eee => generated 0 bytes in 643 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:32:08.091149 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b10310-f2c1-4e51-ac7b-d774e858908a None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:08.091473 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b10310-f2c1-4e51-ac7b-d774e858908a None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:08.091750 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b10310-f2c1-4e51-ac7b-d774e858908a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:08.158798 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-aeba4385-05e0-437d-8c2f-af77ed76477c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fb7498643b84db4882816b5e1931b92', 'request_id': 'req-aeba4385-05e0-437d-8c2f-af77ed76477c'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:08.159837 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-f2b10310-f2c1-4e51-ac7b-d774e858908a None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:32:08.161481 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-f2b10310-f2c1-4e51-ac7b-d774e858908a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:08.172291 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-aeba4385-05e0-437d-8c2f-af77ed76477c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fb7498643b84db4882816b5e1931b92', 'request_id': 'req-aeba4385-05e0-437d-8c2f-af77ed76477c'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:08.247565 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-aeba4385-05e0-437d-8c2f-af77ed76477c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fb7498643b84db4882816b5e1931b92', 'request_id': 'req-aeba4385-05e0-437d-8c2f-af77ed76477c'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:08.263956 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-aeba4385-05e0-437d-8c2f-af77ed76477c admin admin] The token cache is being invalidate because project 1fb7498643b84db4882816b5e1931b92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:32:08.264255 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-aeba4385-05e0-437d-8c2f-af77ed76477c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:08.269193 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-aeba4385-05e0-437d-8c2f-af77ed76477c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:08.269193 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1235/2438] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:32:07 2022] DELETE /identity/v3/projects/1fb7498643b84db4882816b5e1931b92 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:32:08.271253 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:08.272508 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f108d67-5f72-4d67-bb94-a4cd43590bd0 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:08.272866 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f108d67-5f72-4d67-bb94-a4cd43590bd0 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:08.273222 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f108d67-5f72-4d67-bb94-a4cd43590bd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:08.284245 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1204/2439] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:32:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:32:08.299013 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-9f108d67-5f72-4d67-bb94-a4cd43590bd0 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:32:08.300427 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-9f108d67-5f72-4d67-bb94-a4cd43590bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:08.305066 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b10310-f2c1-4e51-ac7b-d774e858908a None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:08.307837 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-f2b10310-f2c1-4e51-ac7b-d774e858908a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:08.321715 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1236/2440] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:32:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:32:08.332587 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13196fc-86cc-44d6-a098-5f6b84a60f61 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:08.333395 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13196fc-86cc-44d6-a098-5f6b84a60f61 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:08.334775 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13196fc-86cc-44d6-a098-5f6b84a60f61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:08.382714 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-b13196fc-86cc-44d6-a098-5f6b84a60f61 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:32:08.385482 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-b13196fc-86cc-44d6-a098-5f6b84a60f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:08.397488 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1237/2441] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:32:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:32:08.398878 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13196fc-86cc-44d6-a098-5f6b84a60f61 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:08.403964 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b10310-f2c1-4e51-ac7b-d774e858908a None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:08.410373 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865dd138-2848-4188-b71d-a138b59b8461 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:08.420929 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865dd138-2848-4188-b71d-a138b59b8461 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:08.420929 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865dd138-2848-4188-b71d-a138b59b8461 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:08.420929 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865dd138-2848-4188-b71d-a138b59b8461 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:08.421455 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13196fc-86cc-44d6-a098-5f6b84a60f61 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:08.421455 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0dd7fe-5002-443b-a167-80fc7c7a256a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:08.421455 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0dd7fe-5002-443b-a167-80fc7c7a256a admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:08.421455 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0dd7fe-5002-443b-a167-80fc7c7a256a admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:08.421812 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0dd7fe-5002-443b-a167-80fc7c7a256a admin admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:08.422034 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0dd7fe-5002-443b-a167-80fc7c7a256a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:32:08.422490 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0dd7fe-5002-443b-a167-80fc7c7a256a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:08.424323 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0dd7fe-5002-443b-a167-80fc7c7a256a admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:08.443603 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865dd138-2848-4188-b71d-a138b59b8461 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:08.445514 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865dd138-2848-4188-b71d-a138b59b8461 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:08.450528 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1238/2442] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Aug 4 21:32:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:32:08.451592 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0dd7fe-5002-443b-a167-80fc7c7a256a admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:08.460473 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1205/2443] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:32:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:32:08.466826 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0dd7fe-5002-443b-a167-80fc7c7a256a admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:08.473734 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137da536-622e-48fe-88f7-f67e506cfaf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:08.474708 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137da536-622e-48fe-88f7-f67e506cfaf4 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:08.474993 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137da536-622e-48fe-88f7-f67e506cfaf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:08.475269 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137da536-622e-48fe-88f7-f67e506cfaf4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:08.475989 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137da536-622e-48fe-88f7-f67e506cfaf4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:08.478422 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137da536-622e-48fe-88f7-f67e506cfaf4 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:08.602400 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1239/2444] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:32:08 2022] POST /identity/v3/projects => generated 391 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:32:08.603001 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865dd138-2848-4188-b71d-a138b59b8461 service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:08.612366 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865dd138-2848-4188-b71d-a138b59b8461 service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:08.685163 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137da536-622e-48fe-88f7-f67e506cfaf4 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:08.687743 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-137da536-622e-48fe-88f7-f67e506cfaf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:08.717976 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646b6234-b888-4337-9ce2-54eca0e673ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:08.718707 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646b6234-b888-4337-9ce2-54eca0e673ec admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:08.719097 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646b6234-b888-4337-9ce2-54eca0e673ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:08.719505 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646b6234-b888-4337-9ce2-54eca0e673ec admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:08.720248 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646b6234-b888-4337-9ce2-54eca0e673ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:08.722609 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646b6234-b888-4337-9ce2-54eca0e673ec admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:08.770182 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137da536-622e-48fe-88f7-f67e506cfaf4 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:08.776200 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5bbf9b-f444-487f-bd73-dd965f03bef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:08.777174 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5bbf9b-f444-487f-bd73-dd965f03bef7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:08.777361 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5bbf9b-f444-487f-bd73-dd965f03bef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:08.777698 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5bbf9b-f444-487f-bd73-dd965f03bef7 admin admin] PATH_INFO: `/v3/projects/20b096f3bc2943e8b662a6745ac25a00` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:08.778950 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5bbf9b-f444-487f-bd73-dd965f03bef7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20b096f3bc2943e8b662a6745ac25a00)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:08.780893 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5bbf9b-f444-487f-bd73-dd965f03bef7 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:08.814912 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1206/2445] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:32:08 2022] POST /identity/v3/users => generated 296 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:32:08.818700 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646b6234-b888-4337-9ce2-54eca0e673ec admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:08.826604 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-646b6234-b888-4337-9ce2-54eca0e673ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:08.839857 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646b6234-b888-4337-9ce2-54eca0e673ec admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:08.844781 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79d899c-eaaa-4031-85d6-bf89d407f8df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:08.845436 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79d899c-eaaa-4031-85d6-bf89d407f8df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:08.845719 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79d899c-eaaa-4031-85d6-bf89d407f8df admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:08.846003 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79d899c-eaaa-4031-85d6-bf89d407f8df admin admin] PATH_INFO: `/v3/projects/5603eb09b4be40b98fe3bad6c40680b7` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:08.847186 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79d899c-eaaa-4031-85d6-bf89d407f8df admin admin] RBAC: Authorizing `identity:delete_project(project_id=5603eb09b4be40b98fe3bad6c40680b7)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:08.849191 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79d899c-eaaa-4031-85d6-bf89d407f8df admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:08.911392 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-0c5bbf9b-f444-487f-bd73-dd965f03bef7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20b096f3bc2943e8b662a6745ac25a00', 'request_id': 'req-0c5bbf9b-f444-487f-bd73-dd965f03bef7'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:08.919351 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-0c5bbf9b-f444-487f-bd73-dd965f03bef7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20b096f3bc2943e8b662a6745ac25a00', 'request_id': 'req-0c5bbf9b-f444-487f-bd73-dd965f03bef7'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:08.978576 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-b79d899c-eaaa-4031-85d6-bf89d407f8df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5603eb09b4be40b98fe3bad6c40680b7', 'request_id': 'req-b79d899c-eaaa-4031-85d6-bf89d407f8df'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:08.986102 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-b79d899c-eaaa-4031-85d6-bf89d407f8df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5603eb09b4be40b98fe3bad6c40680b7', 'request_id': 'req-b79d899c-eaaa-4031-85d6-bf89d407f8df'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:09.072809 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-0c5bbf9b-f444-487f-bd73-dd965f03bef7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20b096f3bc2943e8b662a6745ac25a00', 'request_id': 'req-0c5bbf9b-f444-487f-bd73-dd965f03bef7'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:09.087095 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-0c5bbf9b-f444-487f-bd73-dd965f03bef7 admin admin] The token cache is being invalidate because project 20b096f3bc2943e8b662a6745ac25a00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:32:09.087417 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-0c5bbf9b-f444-487f-bd73-dd965f03bef7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:09.088350 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-0c5bbf9b-f444-487f-bd73-dd965f03bef7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:09.089905 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1240/2446] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:32:08 2022] DELETE /identity/v3/projects/20b096f3bc2943e8b662a6745ac25a00 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:32:09.090807 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5bbf9b-f444-487f-bd73-dd965f03bef7 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:09.092130 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-0c5bbf9b-f444-487f-bd73-dd965f03bef7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:09.101302 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=2 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:09.132806 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-b79d899c-eaaa-4031-85d6-bf89d407f8df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5603eb09b4be40b98fe3bad6c40680b7', 'request_id': 'req-b79d899c-eaaa-4031-85d6-bf89d407f8df'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:09.145885 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-b79d899c-eaaa-4031-85d6-bf89d407f8df admin admin] The token cache is being invalidate because project 5603eb09b4be40b98fe3bad6c40680b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:32:09.146242 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-b79d899c-eaaa-4031-85d6-bf89d407f8df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:09.146865 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-b79d899c-eaaa-4031-85d6-bf89d407f8df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:09.148492 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1207/2447] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:32:08 2022] DELETE /identity/v3/projects/5603eb09b4be40b98fe3bad6c40680b7 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:32:09.152819 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79d899c-eaaa-4031-85d6-bf89d407f8df admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:09.154230 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-b79d899c-eaaa-4031-85d6-bf89d407f8df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:09.176091 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5bbf9b-f444-487f-bd73-dd965f03bef7 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:09.248543 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79d899c-eaaa-4031-85d6-bf89d407f8df admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:09.252394 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602bee0a-d9fe-4543-82bb-2db1ed235781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:09.253941 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602bee0a-d9fe-4543-82bb-2db1ed235781 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:09.255932 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602bee0a-d9fe-4543-82bb-2db1ed235781 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:09.256302 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602bee0a-d9fe-4543-82bb-2db1ed235781 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:09.257762 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af16b68f-41be-4c9c-b30e-46844a18fa12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:09.258160 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602bee0a-d9fe-4543-82bb-2db1ed235781 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:09.258491 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af16b68f-41be-4c9c-b30e-46844a18fa12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:09.258783 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af16b68f-41be-4c9c-b30e-46844a18fa12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:09.260008 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602bee0a-d9fe-4543-82bb-2db1ed235781 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:09.260202 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af16b68f-41be-4c9c-b30e-46844a18fa12 admin admin] PATH_INFO: `/v3/users/f6d553a59f8c43d3b65513c4c7b58c70` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:09.262738 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af16b68f-41be-4c9c-b30e-46844a18fa12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6d553a59f8c43d3b65513c4c7b58c70)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:09.270779 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af16b68f-41be-4c9c-b30e-46844a18fa12 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:09.271922 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1241/2448] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:32:09 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:32:09.273598 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba8ded2-f259-4ab9-b975-635cb770f78d None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:09.273970 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba8ded2-f259-4ab9-b975-635cb770f78d None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:09.274470 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba8ded2-f259-4ab9-b975-635cb770f78d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:09.312917 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-1ba8ded2-f259-4ab9-b975-635cb770f78d None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:32:09.314299 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-1ba8ded2-f259-4ab9-b975-635cb770f78d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:09.325074 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1242/2449] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:32:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:32:09.326109 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba8ded2-f259-4ab9-b975-635cb770f78d None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:09.328824 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-1ba8ded2-f259-4ab9-b975-635cb770f78d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:09.344229 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba8ded2-f259-4ab9-b975-635cb770f78d None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:09.349856 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200f49d5-588d-4b39-9260-b2b9417115f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:09.350186 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200f49d5-588d-4b39-9260-b2b9417115f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:09.350355 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200f49d5-588d-4b39-9260-b2b9417115f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:09.350627 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200f49d5-588d-4b39-9260-b2b9417115f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:09.351344 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200f49d5-588d-4b39-9260-b2b9417115f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:09.352945 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200f49d5-588d-4b39-9260-b2b9417115f4 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:09.364823 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1243/2450] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:32:09 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:32:09.368627 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6703f6-2ea1-4d35-8c05-2e525b3f318e None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:09.368925 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6703f6-2ea1-4d35-8c05-2e525b3f318e None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:09.369228 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6703f6-2ea1-4d35-8c05-2e525b3f318e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:09.388924 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-1d6703f6-2ea1-4d35-8c05-2e525b3f318e None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:32:09.390142 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-1d6703f6-2ea1-4d35-8c05-2e525b3f318e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:09.401248 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1244/2451] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:32:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:32:09.405916 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6703f6-2ea1-4d35-8c05-2e525b3f318e None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:09.412231 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6703f6-2ea1-4d35-8c05-2e525b3f318e None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:09.418158 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b7d4c4-e815-4ebb-9736-e27147844070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:09.418962 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b7d4c4-e815-4ebb-9736-e27147844070 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:09.419584 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b7d4c4-e815-4ebb-9736-e27147844070 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:09.419974 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b7d4c4-e815-4ebb-9736-e27147844070 admin admin] PATH_INFO: `/v3/projects/acb7a28d05334925984cd240d9c7c1a5/users/b8e792178d2a45e980cd615e6e073b5c/roles/8feebc60363c4feb870718b1721e5324` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:09.436667 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b7d4c4-e815-4ebb-9736-e27147844070 admin admin] RBAC: Authorizing `identity:create_grant(project_id=acb7a28d05334925984cd240d9c7c1a5, user_id=b8e792178d2a45e980cd615e6e073b5c, role_id=8feebc60363c4feb870718b1721e5324)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:09.439490 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b7d4c4-e815-4ebb-9736-e27147844070 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:09.503920 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-af16b68f-41be-4c9c-b30e-46844a18fa12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6d553a59f8c43d3b65513c4c7b58c70'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:09.512242 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-af16b68f-41be-4c9c-b30e-46844a18fa12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6d553a59f8c43d3b65513c4c7b58c70'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:09.512962 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-af16b68f-41be-4c9c-b30e-46844a18fa12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6d553a59f8c43d3b65513c4c7b58c70'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:09.520740 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-af16b68f-41be-4c9c-b30e-46844a18fa12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6d553a59f8c43d3b65513c4c7b58c70'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:09.521390 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-af16b68f-41be-4c9c-b30e-46844a18fa12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6d553a59f8c43d3b65513c4c7b58c70'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:09.530233 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1245/2452] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:32:09 2022] PUT /identity/v3/projects/acb7a28d05334925984cd240d9c7c1a5/users/b8e792178d2a45e980cd615e6e073b5c/roles/8feebc60363c4feb870718b1721e5324 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:32:09.538160 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b7d4c4-e815-4ebb-9736-e27147844070 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:09.538160 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-d1b7d4c4-e815-4ebb-9736-e27147844070 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:09.605371 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b7d4c4-e815-4ebb-9736-e27147844070 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:09.610632 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625a990b-27db-436b-b9c0-77112b1c60c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:09.611252 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625a990b-27db-436b-b9c0-77112b1c60c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:09.611580 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625a990b-27db-436b-b9c0-77112b1c60c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:09.611858 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625a990b-27db-436b-b9c0-77112b1c60c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:09.612638 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625a990b-27db-436b-b9c0-77112b1c60c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:32:09.612950 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625a990b-27db-436b-b9c0-77112b1c60c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:09.614568 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625a990b-27db-436b-b9c0-77112b1c60c4 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:09.627177 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1246/2453] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Aug 4 21:32:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:32:09.628005 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1208/2454] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:32:09 2022] DELETE /identity/v3/users/f6d553a59f8c43d3b65513c4c7b58c70 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:32:09.638998 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a88033-7b2d-4948-bb3b-f37ec6718e51 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:09.638998 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a88033-7b2d-4948-bb3b-f37ec6718e51 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:09.638998 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a88033-7b2d-4948-bb3b-f37ec6718e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:09.639869 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af16b68f-41be-4c9c-b30e-46844a18fa12 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:09.676688 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-95a88033-7b2d-4948-bb3b-f37ec6718e51 None None] MFA Rules not processed for user `b8e792178d2a45e980cd615e6e073b5c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:32:09.677993 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-95a88033-7b2d-4948-bb3b-f37ec6718e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:09.710667 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af16b68f-41be-4c9c-b30e-46844a18fa12 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:09.716649 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8873092-66c7-4ff8-b481-cb85f167d40a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:09.717384 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8873092-66c7-4ff8-b481-cb85f167d40a admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:09.717803 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8873092-66c7-4ff8-b481-cb85f167d40a admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:09.718134 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8873092-66c7-4ff8-b481-cb85f167d40a admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:09.719428 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8873092-66c7-4ff8-b481-cb85f167d40a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:09.721536 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8873092-66c7-4ff8-b481-cb85f167d40a admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:09.761405 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1247/2455] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:32:09 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:32:09.765866 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a88033-7b2d-4948-bb3b-f37ec6718e51 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:09.767237 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-95a88033-7b2d-4948-bb3b-f37ec6718e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:09.780596 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a88033-7b2d-4948-bb3b-f37ec6718e51 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:09.786487 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d151d74-da7c-408a-8d89-f0e9613c909f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:09.787288 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d151d74-da7c-408a-8d89-f0e9613c909f admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:09.787611 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d151d74-da7c-408a-8d89-f0e9613c909f admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:09.787889 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d151d74-da7c-408a-8d89-f0e9613c909f admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:09.788566 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d151d74-da7c-408a-8d89-f0e9613c909f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:09.790482 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d151d74-da7c-408a-8d89-f0e9613c909f admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:09.823543 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1209/2456] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:32:09 2022] POST /identity/v3/projects => generated 377 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:32:09.824986 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8873092-66c7-4ff8-b481-cb85f167d40a admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:09.889352 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8873092-66c7-4ff8-b481-cb85f167d40a admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:09.896203 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1248/2457] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:32:09 2022] POST /identity/v3/projects => generated 391 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:32:09.897336 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d151d74-da7c-408a-8d89-f0e9613c909f admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:09.963663 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccb951d-563f-4a8e-a2fd-24d4b0855f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:09.964209 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccb951d-563f-4a8e-a2fd-24d4b0855f71 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:09.964487 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccb951d-563f-4a8e-a2fd-24d4b0855f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:09.964755 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccb951d-563f-4a8e-a2fd-24d4b0855f71 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:09.965599 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccb951d-563f-4a8e-a2fd-24d4b0855f71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:09.967717 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccb951d-563f-4a8e-a2fd-24d4b0855f71 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:10.000288 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d151d74-da7c-408a-8d89-f0e9613c909f admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:10.005320 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d0190b-bded-44bb-ac4d-69c812147264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:10.005945 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d0190b-bded-44bb-ac4d-69c812147264 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:10.006229 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d0190b-bded-44bb-ac4d-69c812147264 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:10.006524 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d0190b-bded-44bb-ac4d-69c812147264 admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:10.007229 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d0190b-bded-44bb-ac4d-69c812147264 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:10.009205 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d0190b-bded-44bb-ac4d-69c812147264 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:10.087279 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1210/2458] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:32:09 2022] POST /identity/v3/users => generated 289 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:32:10.093752 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccb951d-563f-4a8e-a2fd-24d4b0855f71 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:10.105194 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccb951d-563f-4a8e-a2fd-24d4b0855f71 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:10.111235 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f76c6a7-fdcf-4827-b283-dcc66f408569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:10.111974 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f76c6a7-fdcf-4827-b283-dcc66f408569 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:10.112259 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f76c6a7-fdcf-4827-b283-dcc66f408569 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:10.112536 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f76c6a7-fdcf-4827-b283-dcc66f408569 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:10.113376 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f76c6a7-fdcf-4827-b283-dcc66f408569 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:10.115001 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f76c6a7-fdcf-4827-b283-dcc66f408569 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:10.126269 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1211/2459] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:32:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:32:10.130292 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f76c6a7-fdcf-4827-b283-dcc66f408569 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:10.142965 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f76c6a7-fdcf-4827-b283-dcc66f408569 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:10.148669 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a773de4-a06f-44f9-899d-18154c45de57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:10.149300 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a773de4-a06f-44f9-899d-18154c45de57 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:10.149585 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a773de4-a06f-44f9-899d-18154c45de57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:10.149863 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a773de4-a06f-44f9-899d-18154c45de57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:10.151361 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a773de4-a06f-44f9-899d-18154c45de57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:10.158643 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a773de4-a06f-44f9-899d-18154c45de57 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:10.163969 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1249/2460] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:32:09 2022] POST /identity/v3/users => generated 302 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:32:10.165935 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1212/2461] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:32:10 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:32:10.170125 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d0190b-bded-44bb-ac4d-69c812147264 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:10.171152 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a773de4-a06f-44f9-899d-18154c45de57 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:10.181396 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d0190b-bded-44bb-ac4d-69c812147264 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:10.186903 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f25edb-bde1-4993-9117-a9915be62498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:10.187914 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f25edb-bde1-4993-9117-a9915be62498 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:10.188207 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f25edb-bde1-4993-9117-a9915be62498 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:10.188864 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f25edb-bde1-4993-9117-a9915be62498 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:10.189236 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f25edb-bde1-4993-9117-a9915be62498 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:10.191324 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f25edb-bde1-4993-9117-a9915be62498 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:10.193847 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a773de4-a06f-44f9-899d-18154c45de57 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:10.199196 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fcc418-4e94-49bd-a4c8-af20676551a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:10.200021 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fcc418-4e94-49bd-a4c8-af20676551a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:10.200369 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fcc418-4e94-49bd-a4c8-af20676551a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:10.200718 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fcc418-4e94-49bd-a4c8-af20676551a2 admin admin] PATH_INFO: `/v3/projects/5b54066610f6483aa66bc27b5b2de6fe/users/b665d491470f4a0b95c13f51e19d2046/roles/8feebc60363c4feb870718b1721e5324` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:10.206760 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1250/2462] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:32:10 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:32:10.210793 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f25edb-bde1-4993-9117-a9915be62498 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:10.220294 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fcc418-4e94-49bd-a4c8-af20676551a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b54066610f6483aa66bc27b5b2de6fe, user_id=b665d491470f4a0b95c13f51e19d2046, role_id=8feebc60363c4feb870718b1721e5324)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:10.223074 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fcc418-4e94-49bd-a4c8-af20676551a2 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:10.229309 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f25edb-bde1-4993-9117-a9915be62498 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:10.234532 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc354fc-d7de-4937-adf3-1766ada6d802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:10.235231 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc354fc-d7de-4937-adf3-1766ada6d802 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:10.235541 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc354fc-d7de-4937-adf3-1766ada6d802 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:10.235822 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc354fc-d7de-4937-adf3-1766ada6d802 admin admin] PATH_INFO: `/v3/projects/d28d7ee4c45747f685386ad235e48ff4/users/95592a4a336649a9a859836daaf77ddf/roles/a432dd4735944ecfab2fa5ce1705d356` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:10.251846 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc354fc-d7de-4937-adf3-1766ada6d802 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d28d7ee4c45747f685386ad235e48ff4, user_id=95592a4a336649a9a859836daaf77ddf, role_id=a432dd4735944ecfab2fa5ce1705d356)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:10.254632 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc354fc-d7de-4937-adf3-1766ada6d802 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:10.258617 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:10.323719 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1213/2463] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:32:10 2022] PUT /identity/v3/projects/5b54066610f6483aa66bc27b5b2de6fe/users/b665d491470f4a0b95c13f51e19d2046/roles/8feebc60363c4feb870718b1721e5324 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:32:10.323719 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fcc418-4e94-49bd-a4c8-af20676551a2 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:10.325706 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-15fcc418-4e94-49bd-a4c8-af20676551a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:10.367733 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1251/2464] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:32:10 2022] PUT /identity/v3/projects/d28d7ee4c45747f685386ad235e48ff4/users/95592a4a336649a9a859836daaf77ddf/roles/a432dd4735944ecfab2fa5ce1705d356 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:32:10.370640 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485d28c6-046a-4450-8b61-45120418b970 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:10.370640 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485d28c6-046a-4450-8b61-45120418b970 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:10.370640 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485d28c6-046a-4450-8b61-45120418b970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:10.389334 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-485d28c6-046a-4450-8b61-45120418b970 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:32:10.390488 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-485d28c6-046a-4450-8b61-45120418b970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:10.410430 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fcc418-4e94-49bd-a4c8-af20676551a2 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:10.478506 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1252/2465] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:32:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:32:10.478506 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485d28c6-046a-4450-8b61-45120418b970 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:10.486739 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485d28c6-046a-4450-8b61-45120418b970 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:10.492159 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f83269c-9d94-47ba-8dd8-cd8ae7366392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:10.492902 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f83269c-9d94-47ba-8dd8-cd8ae7366392 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:10.493287 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f83269c-9d94-47ba-8dd8-cd8ae7366392 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:10.493883 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f83269c-9d94-47ba-8dd8-cd8ae7366392 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:10.494850 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f83269c-9d94-47ba-8dd8-cd8ae7366392 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:10.500706 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f83269c-9d94-47ba-8dd8-cd8ae7366392 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:10.511368 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1253/2466] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:32:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:32:10.511368 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f83269c-9d94-47ba-8dd8-cd8ae7366392 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:10.511368 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-0f83269c-9d94-47ba-8dd8-cd8ae7366392 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:10.518439 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad54e4d9-923c-45d9-a30b-b34a236aa3b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:10.524588 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad54e4d9-923c-45d9-a30b-b34a236aa3b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:10.524588 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad54e4d9-923c-45d9-a30b-b34a236aa3b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:10.524588 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad54e4d9-923c-45d9-a30b-b34a236aa3b0 admin admin] PATH_INFO: `/v3/projects/b9b793168cc44baa9cce816ee861e197` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:10.524588 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad54e4d9-923c-45d9-a30b-b34a236aa3b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9b793168cc44baa9cce816ee861e197)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:10.524588 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad54e4d9-923c-45d9-a30b-b34a236aa3b0 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:10.580091 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f83269c-9d94-47ba-8dd8-cd8ae7366392 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:10.584448 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47beda17-29d6-432b-87b7-c10a4a9140d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:10.585669 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47beda17-29d6-432b-87b7-c10a4a9140d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:10.585962 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47beda17-29d6-432b-87b7-c10a4a9140d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:10.586255 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47beda17-29d6-432b-87b7-c10a4a9140d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:10.596701 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47beda17-29d6-432b-87b7-c10a4a9140d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:10.598170 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47beda17-29d6-432b-87b7-c10a4a9140d0 service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:10.608444 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1254/2467] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:32:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:32:10.609405 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47beda17-29d6-432b-87b7-c10a4a9140d0 service neutron] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:10.632522 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47beda17-29d6-432b-87b7-c10a4a9140d0 service neutron] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:10.632522 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fde76a1-3e8c-4d3e-9b10-52aab8412d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:10.632522 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde76a1-3e8c-4d3e-9b10-52aab8412d00 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:10.632522 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde76a1-3e8c-4d3e-9b10-52aab8412d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:10.633254 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde76a1-3e8c-4d3e-9b10-52aab8412d00 admin admin] PATH_INFO: `/v3/projects/d28d7ee4c45747f685386ad235e48ff4/users/95592a4a336649a9a859836daaf77ddf/roles/a432dd4735944ecfab2fa5ce1705d356` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:10.633254 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fde76a1-3e8c-4d3e-9b10-52aab8412d00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d28d7ee4c45747f685386ad235e48ff4, user_id=95592a4a336649a9a859836daaf77ddf, role_id=a432dd4735944ecfab2fa5ce1705d356)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:10.633254 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fde76a1-3e8c-4d3e-9b10-52aab8412d00 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:10.643166 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1255/2468] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:32:10 2022] PUT /identity/v3/projects/d28d7ee4c45747f685386ad235e48ff4/users/95592a4a336649a9a859836daaf77ddf/roles/a432dd4735944ecfab2fa5ce1705d356 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:32:10.653271 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fde3b63-a904-4765-8385-b9533577294b None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:10.653589 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fde3b63-a904-4765-8385-b9533577294b None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:10.654184 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fde3b63-a904-4765-8385-b9533577294b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:10.675270 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ad54e4d9-923c-45d9-a30b-b34a236aa3b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9b793168cc44baa9cce816ee861e197', 'request_id': 'req-ad54e4d9-923c-45d9-a30b-b34a236aa3b0'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:10.681100 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ad54e4d9-923c-45d9-a30b-b34a236aa3b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9b793168cc44baa9cce816ee861e197', 'request_id': 'req-ad54e4d9-923c-45d9-a30b-b34a236aa3b0'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:10.685064 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-4fde3b63-a904-4765-8385-b9533577294b None None] MFA Rules not processed for user `95592a4a336649a9a859836daaf77ddf`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:32:10.686319 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-4fde3b63-a904-4765-8385-b9533577294b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:10.760871 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1256/2469] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:32:10 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:32:10.770489 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ad54e4d9-923c-45d9-a30b-b34a236aa3b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9b793168cc44baa9cce816ee861e197', 'request_id': 'req-ad54e4d9-923c-45d9-a30b-b34a236aa3b0'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:10.772170 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fde3b63-a904-4765-8385-b9533577294b None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:10.773599 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-4fde3b63-a904-4765-8385-b9533577294b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:10.787875 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ad54e4d9-923c-45d9-a30b-b34a236aa3b0 admin admin] The token cache is being invalidate because project b9b793168cc44baa9cce816ee861e197 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:32:10.788535 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ad54e4d9-923c-45d9-a30b-b34a236aa3b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:10.789237 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ad54e4d9-923c-45d9-a30b-b34a236aa3b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:10.791661 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1214/2470] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:32:10 2022] DELETE /identity/v3/projects/b9b793168cc44baa9cce816ee861e197 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:32:10.796366 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b07b87-acfb-4712-8323-162291bfac4d None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:10.797792 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b07b87-acfb-4712-8323-162291bfac4d None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:10.797959 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b07b87-acfb-4712-8323-162291bfac4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:10.820632 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-f0b07b87-acfb-4712-8323-162291bfac4d None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:32:10.821939 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-f0b07b87-acfb-4712-8323-162291bfac4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:10.841288 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fde3b63-a904-4765-8385-b9533577294b None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:10.847556 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9908287-7413-409a-90b3-09ea0b80b95c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e31f2728056b460fa474478eb85ca1d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:10.848226 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9908287-7413-409a-90b3-09ea0b80b95c service cinder] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:10.848444 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9908287-7413-409a-90b3-09ea0b80b95c service cinder] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:10.848715 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9908287-7413-409a-90b3-09ea0b80b95c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:10.850663 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-f9908287-7413-409a-90b3-09ea0b80b95c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:10.864001 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9908287-7413-409a-90b3-09ea0b80b95c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:10.865495 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9908287-7413-409a-90b3-09ea0b80b95c service cinder] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:10.875717 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1257/2471] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:32:10 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:32:10.887474 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1215/2472] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:32:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:32:10.896229 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74369de-ed18-4a56-9a15-afff358259ab None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:10.896598 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74369de-ed18-4a56-9a15-afff358259ab None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:10.896939 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74369de-ed18-4a56-9a15-afff358259ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:10.926991 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b07b87-acfb-4712-8323-162291bfac4d None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:10.928639 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-f0b07b87-acfb-4712-8323-162291bfac4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:10.930647 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-d74369de-ed18-4a56-9a15-afff358259ab None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:32:10.931504 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-d74369de-ed18-4a56-9a15-afff358259ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:10.940971 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1258/2473] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:32:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:32:10.945886 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b07b87-acfb-4712-8323-162291bfac4d None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:10.946245 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74369de-ed18-4a56-9a15-afff358259ab None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:10.950038 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36ee202-1bbd-4725-b998-2f837325a50d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e31f2728056b460fa474478eb85ca1d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:10.950941 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36ee202-1bbd-4725-b998-2f837325a50d service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:10.951230 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36ee202-1bbd-4725-b998-2f837325a50d service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:10.951527 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36ee202-1bbd-4725-b998-2f837325a50d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:10.954260 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-b36ee202-1bbd-4725-b998-2f837325a50d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:10.968509 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74369de-ed18-4a56-9a15-afff358259ab None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:10.972170 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:10.976578 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e73e8e-95f4-4c47-9254-c1203c654070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:10.980857 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e73e8e-95f4-4c47-9254-c1203c654070 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:10.980857 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e73e8e-95f4-4c47-9254-c1203c654070 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:10.980857 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e73e8e-95f4-4c47-9254-c1203c654070 admin admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:10.980857 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e73e8e-95f4-4c47-9254-c1203c654070 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:32:10.980857 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e73e8e-95f4-4c47-9254-c1203c654070 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:10.981384 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e73e8e-95f4-4c47-9254-c1203c654070 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:10.994327 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1259/2474] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Aug 4 21:32:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:32:10.998732 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e73e8e-95f4-4c47-9254-c1203c654070 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:11.013004 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e73e8e-95f4-4c47-9254-c1203c654070 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:11.024941 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d034db51-369e-436d-ae73-866a0fce870b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:11.025787 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d034db51-369e-436d-ae73-866a0fce870b admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:11.026142 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d034db51-369e-436d-ae73-866a0fce870b admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:11.026655 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d034db51-369e-436d-ae73-866a0fce870b admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:11.028226 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d034db51-369e-436d-ae73-866a0fce870b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:11.029777 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d034db51-369e-436d-ae73-866a0fce870b admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:11.030738 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36ee202-1bbd-4725-b998-2f837325a50d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:11.033043 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36ee202-1bbd-4725-b998-2f837325a50d service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:11.043679 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1216/2475] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:32:10 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:32:11.119385 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1260/2476] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:32:10 2022] POST /identity/v3/projects => generated 367 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:32:11.121646 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36ee202-1bbd-4725-b998-2f837325a50d service cinder] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:11.135572 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36ee202-1bbd-4725-b998-2f837325a50d service cinder] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:11.210336 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b34861-9ba8-438a-92b3-b7797c22862e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:11.211082 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b34861-9ba8-438a-92b3-b7797c22862e admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:11.211477 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b34861-9ba8-438a-92b3-b7797c22862e admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:11.211819 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b34861-9ba8-438a-92b3-b7797c22862e admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:11.212557 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b34861-9ba8-438a-92b3-b7797c22862e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:11.214578 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b34861-9ba8-438a-92b3-b7797c22862e admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:11.365061 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1217/2477] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:32:11 2022] POST /identity/v3/users => generated 284 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:32:11.369277 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d034db51-369e-436d-ae73-866a0fce870b admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:11.380490 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d034db51-369e-436d-ae73-866a0fce870b admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:11.385078 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773f853e-70f5-43d9-a9a1-a6d2729217b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:11.385632 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773f853e-70f5-43d9-a9a1-a6d2729217b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:11.385911 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773f853e-70f5-43d9-a9a1-a6d2729217b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:11.386180 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773f853e-70f5-43d9-a9a1-a6d2729217b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:11.386988 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773f853e-70f5-43d9-a9a1-a6d2729217b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:11.388639 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773f853e-70f5-43d9-a9a1-a6d2729217b1 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:11.399991 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1261/2478] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:32:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:32:11.404885 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b34861-9ba8-438a-92b3-b7797c22862e admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:11.415876 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b34861-9ba8-438a-92b3-b7797c22862e admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:11.421286 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e4d0fe-c1c2-4e28-8fa5-2ff53dcc382b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:11.421904 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e4d0fe-c1c2-4e28-8fa5-2ff53dcc382b admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:11.422186 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e4d0fe-c1c2-4e28-8fa5-2ff53dcc382b admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:11.422461 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e4d0fe-c1c2-4e28-8fa5-2ff53dcc382b admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:11.423155 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e4d0fe-c1c2-4e28-8fa5-2ff53dcc382b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:11.424819 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e4d0fe-c1c2-4e28-8fa5-2ff53dcc382b admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:11.435210 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1218/2479] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:32:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:32:11.440112 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773f853e-70f5-43d9-a9a1-a6d2729217b1 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:11.450601 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773f853e-70f5-43d9-a9a1-a6d2729217b1 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:11.457038 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c35458-85d3-4227-87ae-8fbd724ced36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:11.457730 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c35458-85d3-4227-87ae-8fbd724ced36 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:11.457993 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c35458-85d3-4227-87ae-8fbd724ced36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:11.458262 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c35458-85d3-4227-87ae-8fbd724ced36 admin admin] PATH_INFO: `/v3/projects/301dd99bc8f048f7abfda53b2f5fbd67/users/6b71b044d7bd41a983930004ad8a1751/roles/8feebc60363c4feb870718b1721e5324` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:11.474057 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c35458-85d3-4227-87ae-8fbd724ced36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=301dd99bc8f048f7abfda53b2f5fbd67, user_id=6b71b044d7bd41a983930004ad8a1751, role_id=8feebc60363c4feb870718b1721e5324)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:11.476838 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c35458-85d3-4227-87ae-8fbd724ced36 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:11.592901 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1262/2480] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:32:11 2022] PUT /identity/v3/projects/301dd99bc8f048f7abfda53b2f5fbd67/users/6b71b044d7bd41a983930004ad8a1751/roles/8feebc60363c4feb870718b1721e5324 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:32:11.604118 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a48275c-8a91-4f66-8723-2b4f2a1629f0 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:11.604630 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a48275c-8a91-4f66-8723-2b4f2a1629f0 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:11.604735 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a48275c-8a91-4f66-8723-2b4f2a1629f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:11.632061 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-4a48275c-8a91-4f66-8723-2b4f2a1629f0 None None] MFA Rules not processed for user `6b71b044d7bd41a983930004ad8a1751`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:32:11.633283 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-4a48275c-8a91-4f66-8723-2b4f2a1629f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:11.710035 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1219/2481] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:32:11 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:32:11.714548 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c35458-85d3-4227-87ae-8fbd724ced36 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:11.776275 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:11.789068 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c35458-85d3-4227-87ae-8fbd724ced36 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:11.794706 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a470ff33-d0d7-4245-b937-5a350524072b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:11.795344 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a470ff33-d0d7-4245-b937-5a350524072b admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:11.795651 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a470ff33-d0d7-4245-b937-5a350524072b admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:11.795926 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a470ff33-d0d7-4245-b937-5a350524072b admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:11.796643 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a470ff33-d0d7-4245-b937-5a350524072b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:11.799253 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a470ff33-d0d7-4245-b937-5a350524072b admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:11.898758 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1263/2482] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:32:11 2022] POST /identity/v3/projects => generated 367 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:32:11.903048 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a48275c-8a91-4f66-8723-2b4f2a1629f0 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:11.914511 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a48275c-8a91-4f66-8723-2b4f2a1629f0 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:11.982454 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc99de4-f74c-465b-9c66-2d6aa95b3361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:11.983483 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc99de4-f74c-465b-9c66-2d6aa95b3361 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:11.983748 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc99de4-f74c-465b-9c66-2d6aa95b3361 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:11.984025 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc99de4-f74c-465b-9c66-2d6aa95b3361 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:11.984768 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc99de4-f74c-465b-9c66-2d6aa95b3361 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:11.986780 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc99de4-f74c-465b-9c66-2d6aa95b3361 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:12.195524 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1220/2483] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:32:11 2022] POST /identity/v3/users => generated 290 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:32:12.199684 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a470ff33-d0d7-4245-b937-5a350524072b admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:12.211619 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a470ff33-d0d7-4245-b937-5a350524072b admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:12.218148 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89a8c7e-8131-4bce-9024-357fbcd3a0d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:12.218997 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89a8c7e-8131-4bce-9024-357fbcd3a0d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:12.219525 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89a8c7e-8131-4bce-9024-357fbcd3a0d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:12.225041 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89a8c7e-8131-4bce-9024-357fbcd3a0d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:12.225041 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89a8c7e-8131-4bce-9024-357fbcd3a0d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:12.225041 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89a8c7e-8131-4bce-9024-357fbcd3a0d1 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:12.236373 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1264/2484] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:32:12 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:32:12.240754 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc99de4-f74c-465b-9c66-2d6aa95b3361 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:12.252762 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc99de4-f74c-465b-9c66-2d6aa95b3361 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:12.257665 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18eae24-5027-4fc5-8fb4-cfe947f3f51e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:12.258365 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18eae24-5027-4fc5-8fb4-cfe947f3f51e admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:12.259452 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18eae24-5027-4fc5-8fb4-cfe947f3f51e admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:12.259452 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18eae24-5027-4fc5-8fb4-cfe947f3f51e admin admin] PATH_INFO: `/v3/projects/e8e42b51e53b49a68f903fab6aa5d917/users/4bd9b51d4bd6488999b96fd1042efd46/roles/a432dd4735944ecfab2fa5ce1705d356` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:12.278598 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18eae24-5027-4fc5-8fb4-cfe947f3f51e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8e42b51e53b49a68f903fab6aa5d917, user_id=4bd9b51d4bd6488999b96fd1042efd46, role_id=a432dd4735944ecfab2fa5ce1705d356)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:12.282306 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18eae24-5027-4fc5-8fb4-cfe947f3f51e admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:12.377416 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1221/2485] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:32:12 2022] PUT /identity/v3/projects/e8e42b51e53b49a68f903fab6aa5d917/users/4bd9b51d4bd6488999b96fd1042efd46/roles/a432dd4735944ecfab2fa5ce1705d356 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:32:12.381815 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89a8c7e-8131-4bce-9024-357fbcd3a0d1 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:12.395015 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89a8c7e-8131-4bce-9024-357fbcd3a0d1 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:12.456339 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:12.469024 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a355b7f-41dd-4908-b6a5-f39c0a517f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:12.469732 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a355b7f-41dd-4908-b6a5-f39c0a517f7f admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:12.470018 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a355b7f-41dd-4908-b6a5-f39c0a517f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:12.470290 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a355b7f-41dd-4908-b6a5-f39c0a517f7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:12.471153 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a355b7f-41dd-4908-b6a5-f39c0a517f7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:12.473711 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a355b7f-41dd-4908-b6a5-f39c0a517f7f admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:12.484164 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1265/2486] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:32:12 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:32:12.489929 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18eae24-5027-4fc5-8fb4-cfe947f3f51e admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:12.500874 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18eae24-5027-4fc5-8fb4-cfe947f3f51e admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:12.505681 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3fd38c-a37a-4b98-9b7f-96a50eb1c089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:12.506341 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3fd38c-a37a-4b98-9b7f-96a50eb1c089 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:12.506623 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3fd38c-a37a-4b98-9b7f-96a50eb1c089 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:12.506894 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3fd38c-a37a-4b98-9b7f-96a50eb1c089 admin admin] PATH_INFO: `/v3/projects/e8e42b51e53b49a68f903fab6aa5d917/users/4bd9b51d4bd6488999b96fd1042efd46/roles/a432dd4735944ecfab2fa5ce1705d356` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:12.508848 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3fd38c-a37a-4b98-9b7f-96a50eb1c089 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8e42b51e53b49a68f903fab6aa5d917, user_id=4bd9b51d4bd6488999b96fd1042efd46, role_id=a432dd4735944ecfab2fa5ce1705d356)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:12.511575 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3fd38c-a37a-4b98-9b7f-96a50eb1c089 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:12.521925 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1222/2487] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:32:12 2022] PUT /identity/v3/projects/e8e42b51e53b49a68f903fab6aa5d917/users/4bd9b51d4bd6488999b96fd1042efd46/roles/a432dd4735944ecfab2fa5ce1705d356 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:32:12.533313 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c908af9b-c806-4733-ac4a-5017c5734c1b None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:12.533534 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c908af9b-c806-4733-ac4a-5017c5734c1b None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:12.533810 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c908af9b-c806-4733-ac4a-5017c5734c1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:12.561952 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-c908af9b-c806-4733-ac4a-5017c5734c1b None None] MFA Rules not processed for user `4bd9b51d4bd6488999b96fd1042efd46`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:32:12.563988 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-c908af9b-c806-4733-ac4a-5017c5734c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:12.636623 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1266/2488] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:32:12 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:32:12.648338 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3fd38c-a37a-4b98-9b7f-96a50eb1c089 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:12.650308 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-ff3fd38c-a37a-4b98-9b7f-96a50eb1c089 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:12.721619 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3fd38c-a37a-4b98-9b7f-96a50eb1c089 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:12.727497 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c927a82f-bc20-4d5a-899a-3be4504cb947 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e31f2728056b460fa474478eb85ca1d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:12.728370 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c927a82f-bc20-4d5a-899a-3be4504cb947 service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:12.729117 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c927a82f-bc20-4d5a-899a-3be4504cb947 service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:12.729592 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c927a82f-bc20-4d5a-899a-3be4504cb947 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:12.741144 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c927a82f-bc20-4d5a-899a-3be4504cb947 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:12.742966 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c927a82f-bc20-4d5a-899a-3be4504cb947 service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:12.756012 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1223/2489] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:32:12 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:32:12.974011 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c908af9b-c806-4733-ac4a-5017c5734c1b None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:12.989824 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c908af9b-c806-4733-ac4a-5017c5734c1b None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:12.993877 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5af124-92c7-4da3-bbd6-e31a2b868715 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e31f2728056b460fa474478eb85ca1d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:12.995228 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5af124-92c7-4da3-bbd6-e31a2b868715 service cinder] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:12.995834 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5af124-92c7-4da3-bbd6-e31a2b868715 service cinder] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:12.996274 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5af124-92c7-4da3-bbd6-e31a2b868715 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:13.083520 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5af124-92c7-4da3-bbd6-e31a2b868715 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:13.085022 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5af124-92c7-4da3-bbd6-e31a2b868715 service cinder] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:13.098674 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1267/2490] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:32:12 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:32:13.414683 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c927a82f-bc20-4d5a-899a-3be4504cb947 service cinder] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:13.419019 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-c927a82f-bc20-4d5a-899a-3be4504cb947 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:13.491761 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c927a82f-bc20-4d5a-899a-3be4504cb947 service cinder] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:13.495467 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016c3845-a150-469e-8434-c4f7e44f06b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:13.496090 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016c3845-a150-469e-8434-c4f7e44f06b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:13.496387 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016c3845-a150-469e-8434-c4f7e44f06b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:13.496653 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016c3845-a150-469e-8434-c4f7e44f06b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:13.498503 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-016c3845-a150-469e-8434-c4f7e44f06b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:13.544344 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:13.584618 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016c3845-a150-469e-8434-c4f7e44f06b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:13.586091 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016c3845-a150-469e-8434-c4f7e44f06b7 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:13.603510 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1224/2491] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:32:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:32:13.726345 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:14.871429 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:15.222141 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:16.772882 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5af124-92c7-4da3-bbd6-e31a2b868715 service cinder] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:16.774516 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-7b5af124-92c7-4da3-bbd6-e31a2b868715 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:16.808903 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5af124-92c7-4da3-bbd6-e31a2b868715 service cinder] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:16.816756 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8314ff3e-c770-4711-a2aa-9ef57ea82133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:16.823241 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8314ff3e-c770-4711-a2aa-9ef57ea82133 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:16.823241 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8314ff3e-c770-4711-a2aa-9ef57ea82133 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:16.823241 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8314ff3e-c770-4711-a2aa-9ef57ea82133 admin admin] PATH_INFO: `/v3/users/b8e792178d2a45e980cd615e6e073b5c` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:16.823241 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8314ff3e-c770-4711-a2aa-9ef57ea82133 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8e792178d2a45e980cd615e6e073b5c)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:16.823241 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8314ff3e-c770-4711-a2aa-9ef57ea82133 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:17.056665 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cbfe69-c234-47bf-b8d9-0c5e0f6960b7 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:17.057146 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cbfe69-c234-47bf-b8d9-0c5e0f6960b7 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:17.057257 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cbfe69-c234-47bf-b8d9-0c5e0f6960b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:17.085465 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-8314ff3e-c770-4711-a2aa-9ef57ea82133 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8e792178d2a45e980cd615e6e073b5c'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:17.089930 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-f1cbfe69-c234-47bf-b8d9-0c5e0f6960b7 None None] MFA Rules not processed for user `b665d491470f4a0b95c13f51e19d2046`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:32:17.091130 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-f1cbfe69-c234-47bf-b8d9-0c5e0f6960b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:17.102521 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-8314ff3e-c770-4711-a2aa-9ef57ea82133 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8e792178d2a45e980cd615e6e073b5c'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:17.103791 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-8314ff3e-c770-4711-a2aa-9ef57ea82133 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8e792178d2a45e980cd615e6e073b5c'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:17.110302 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-8314ff3e-c770-4711-a2aa-9ef57ea82133 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8e792178d2a45e980cd615e6e073b5c'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:17.174627 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1225/2492] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:32:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:32:17.177152 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cbfe69-c234-47bf-b8d9-0c5e0f6960b7 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:17.178577 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-f1cbfe69-c234-47bf-b8d9-0c5e0f6960b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:17.192944 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cbfe69-c234-47bf-b8d9-0c5e0f6960b7 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:17.198020 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7f08dc-89b4-4709-b794-c901db87af8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:17.198632 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7f08dc-89b4-4709-b794-c901db87af8b admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:17.198918 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7f08dc-89b4-4709-b794-c901db87af8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:17.199222 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7f08dc-89b4-4709-b794-c901db87af8b admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:17.199964 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7f08dc-89b4-4709-b794-c901db87af8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:17.201964 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7f08dc-89b4-4709-b794-c901db87af8b admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:17.230391 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-8314ff3e-c770-4711-a2aa-9ef57ea82133 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8e792178d2a45e980cd615e6e073b5c'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:17.244064 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1268/2493] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:32:16 2022] DELETE /identity/v3/users/b8e792178d2a45e980cd615e6e073b5c => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:32:17.248322 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147971b5-8b8f-41a4-8280-74b76e5b2cdb None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:17.248605 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147971b5-8b8f-41a4-8280-74b76e5b2cdb None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:17.248881 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147971b5-8b8f-41a4-8280-74b76e5b2cdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:17.267174 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-147971b5-8b8f-41a4-8280-74b76e5b2cdb None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:32:17.268391 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-147971b5-8b8f-41a4-8280-74b76e5b2cdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:17.327487 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1226/2494] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:32:17 2022] POST /identity/v3/projects => generated 379 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:32:17.328144 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7f08dc-89b4-4709-b794-c901db87af8b admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:17.335447 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:17.342828 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1269/2495] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:32:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:32:17.353074 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147971b5-8b8f-41a4-8280-74b76e5b2cdb None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:17.354688 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-147971b5-8b8f-41a4-8280-74b76e5b2cdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:17.401268 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7f08dc-89b4-4709-b794-c901db87af8b admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:17.408824 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd186d8b-0097-4984-aec2-f55d8be0bcc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:17.409655 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd186d8b-0097-4984-aec2-f55d8be0bcc8 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:17.410121 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd186d8b-0097-4984-aec2-f55d8be0bcc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:17.411150 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd186d8b-0097-4984-aec2-f55d8be0bcc8 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:17.411858 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd186d8b-0097-4984-aec2-f55d8be0bcc8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:17.414817 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd186d8b-0097-4984-aec2-f55d8be0bcc8 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:17.435701 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147971b5-8b8f-41a4-8280-74b76e5b2cdb None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:17.537778 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1227/2496] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:32:17 2022] POST /identity/v3/users => generated 296 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:32:17.540212 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd186d8b-0097-4984-aec2-f55d8be0bcc8 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:17.554826 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a2058c-8b00-4e7f-9008-8c1bc1db6d37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:17.556207 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a2058c-8b00-4e7f-9008-8c1bc1db6d37 service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:17.556494 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a2058c-8b00-4e7f-9008-8c1bc1db6d37 service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:17.556779 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a2058c-8b00-4e7f-9008-8c1bc1db6d37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:17.574545 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a2058c-8b00-4e7f-9008-8c1bc1db6d37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:17.576062 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a2058c-8b00-4e7f-9008-8c1bc1db6d37 service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:17.578980 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd186d8b-0097-4984-aec2-f55d8be0bcc8 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:17.587916 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621d6822-28bb-43e7-ae57-9c484a6100e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:17.588732 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621d6822-28bb-43e7-ae57-9c484a6100e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:17.589086 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621d6822-28bb-43e7-ae57-9c484a6100e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:17.589437 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621d6822-28bb-43e7-ae57-9c484a6100e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:17.591611 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621d6822-28bb-43e7-ae57-9c484a6100e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:17.593177 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1270/2497] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:32:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:32:17.594599 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621d6822-28bb-43e7-ae57-9c484a6100e8 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:17.638508 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1228/2498] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:32:17 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:32:17.642931 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a2058c-8b00-4e7f-9008-8c1bc1db6d37 service neutron] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:17.659375 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a2058c-8b00-4e7f-9008-8c1bc1db6d37 service neutron] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:17.670937 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5d9a00-8b79-4a9e-97a9-326568cdb969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:17.671762 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5d9a00-8b79-4a9e-97a9-326568cdb969 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:17.672827 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5d9a00-8b79-4a9e-97a9-326568cdb969 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:17.673112 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5d9a00-8b79-4a9e-97a9-326568cdb969 admin admin] PATH_INFO: `/v3/projects/40498e1369524be3b9ef33e69f3539bd/users/93956198082e44189d090d4c5fb95b63/roles/a432dd4735944ecfab2fa5ce1705d356` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:17.698765 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5d9a00-8b79-4a9e-97a9-326568cdb969 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40498e1369524be3b9ef33e69f3539bd, user_id=93956198082e44189d090d4c5fb95b63, role_id=a432dd4735944ecfab2fa5ce1705d356)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:17.698765 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5d9a00-8b79-4a9e-97a9-326568cdb969 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:17.791419 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1271/2499] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:32:17 2022] PUT /identity/v3/projects/40498e1369524be3b9ef33e69f3539bd/users/93956198082e44189d090d4c5fb95b63/roles/a432dd4735944ecfab2fa5ce1705d356 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:32:17.802255 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621d6822-28bb-43e7-ae57-9c484a6100e8 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:17.831764 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621d6822-28bb-43e7-ae57-9c484a6100e8 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:17.853828 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:17.927525 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9654f05a-8838-4fd6-b4eb-6cd5b726a184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:17.928371 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9654f05a-8838-4fd6-b4eb-6cd5b726a184 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:17.928655 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9654f05a-8838-4fd6-b4eb-6cd5b726a184 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:17.928930 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9654f05a-8838-4fd6-b4eb-6cd5b726a184 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:17.932444 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9654f05a-8838-4fd6-b4eb-6cd5b726a184 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:17.932444 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9654f05a-8838-4fd6-b4eb-6cd5b726a184 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:17.969578 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1229/2500] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:32:17 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:32:17.970795 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5d9a00-8b79-4a9e-97a9-326568cdb969 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:18.023230 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5d9a00-8b79-4a9e-97a9-326568cdb969 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:18.033603 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35955514-c46f-4d8e-9300-c62e68c5dd66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:18.036402 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35955514-c46f-4d8e-9300-c62e68c5dd66 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:18.037511 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35955514-c46f-4d8e-9300-c62e68c5dd66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:18.038104 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35955514-c46f-4d8e-9300-c62e68c5dd66 admin admin] PATH_INFO: `/v3/projects/40498e1369524be3b9ef33e69f3539bd/users/93956198082e44189d090d4c5fb95b63/roles/a432dd4735944ecfab2fa5ce1705d356` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:18.040949 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35955514-c46f-4d8e-9300-c62e68c5dd66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40498e1369524be3b9ef33e69f3539bd, user_id=93956198082e44189d090d4c5fb95b63, role_id=a432dd4735944ecfab2fa5ce1705d356)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:18.045009 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35955514-c46f-4d8e-9300-c62e68c5dd66 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:18.082645 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1272/2501] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:32:17 2022] PUT /identity/v3/projects/40498e1369524be3b9ef33e69f3539bd/users/93956198082e44189d090d4c5fb95b63/roles/a432dd4735944ecfab2fa5ce1705d356 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:32:18.319778 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9654f05a-8838-4fd6-b4eb-6cd5b726a184 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:18.322828 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-9654f05a-8838-4fd6-b4eb-6cd5b726a184 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:18.349973 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9654f05a-8838-4fd6-b4eb-6cd5b726a184 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:18.436938 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a450139-9413-4bdb-8383-bc499136740f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:18.437847 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a450139-9413-4bdb-8383-bc499136740f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:18.438322 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a450139-9413-4bdb-8383-bc499136740f admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:18.438771 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a450139-9413-4bdb-8383-bc499136740f admin admin] PATH_INFO: `/v3/projects/acb7a28d05334925984cd240d9c7c1a5` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:18.440502 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a450139-9413-4bdb-8383-bc499136740f admin admin] RBAC: Authorizing `identity:delete_project(project_id=acb7a28d05334925984cd240d9c7c1a5)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:18.442721 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a450139-9413-4bdb-8383-bc499136740f admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:18.583290 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-9a450139-9413-4bdb-8383-bc499136740f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acb7a28d05334925984cd240d9c7c1a5', 'request_id': 'req-9a450139-9413-4bdb-8383-bc499136740f'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:18.612693 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-9a450139-9413-4bdb-8383-bc499136740f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acb7a28d05334925984cd240d9c7c1a5', 'request_id': 'req-9a450139-9413-4bdb-8383-bc499136740f'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:18.738318 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-9a450139-9413-4bdb-8383-bc499136740f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acb7a28d05334925984cd240d9c7c1a5', 'request_id': 'req-9a450139-9413-4bdb-8383-bc499136740f'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:18.754419 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-9a450139-9413-4bdb-8383-bc499136740f admin admin] The token cache is being invalidate because project acb7a28d05334925984cd240d9c7c1a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:32:18.754703 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-9a450139-9413-4bdb-8383-bc499136740f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:18.755674 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-9a450139-9413-4bdb-8383-bc499136740f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:18.757772 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1230/2502] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:32:18 2022] DELETE /identity/v3/projects/acb7a28d05334925984cd240d9c7c1a5 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:32:18.761895 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35955514-c46f-4d8e-9300-c62e68c5dd66 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:18.763994 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-35955514-c46f-4d8e-9300-c62e68c5dd66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:18.783175 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35955514-c46f-4d8e-9300-c62e68c5dd66 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:18.866660 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e7d0f8-a5d4-4e4d-8cd1-aafdc0f3430d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:18.873068 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e7d0f8-a5d4-4e4d-8cd1-aafdc0f3430d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:18.873068 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e7d0f8-a5d4-4e4d-8cd1-aafdc0f3430d admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:18.873068 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e7d0f8-a5d4-4e4d-8cd1-aafdc0f3430d admin admin] PATH_INFO: `/v3/users/95592a4a336649a9a859836daaf77ddf` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:18.873068 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e7d0f8-a5d4-4e4d-8cd1-aafdc0f3430d admin admin] RBAC: Authorizing `identity:delete_user(user_id=95592a4a336649a9a859836daaf77ddf)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:18.873068 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e7d0f8-a5d4-4e4d-8cd1-aafdc0f3430d admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:18.936015 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:19.050664 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a450139-9413-4bdb-8383-bc499136740f admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:19.052183 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-9a450139-9413-4bdb-8383-bc499136740f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:19.066408 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a450139-9413-4bdb-8383-bc499136740f admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:19.076390 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e3799b-b277-4f56-a575-be6eb916cab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:19.077137 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e3799b-b277-4f56-a575-be6eb916cab8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:19.077424 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e3799b-b277-4f56-a575-be6eb916cab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:19.077700 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e3799b-b277-4f56-a575-be6eb916cab8 admin admin] PATH_INFO: `/v3/users/6b71b044d7bd41a983930004ad8a1751` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:19.079160 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e3799b-b277-4f56-a575-be6eb916cab8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b71b044d7bd41a983930004ad8a1751)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:19.081448 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e3799b-b277-4f56-a575-be6eb916cab8 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:19.128284 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-d7e7d0f8-a5d4-4e4d-8cd1-aafdc0f3430d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95592a4a336649a9a859836daaf77ddf'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:19.143687 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-d7e7d0f8-a5d4-4e4d-8cd1-aafdc0f3430d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95592a4a336649a9a859836daaf77ddf'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:19.144439 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-d7e7d0f8-a5d4-4e4d-8cd1-aafdc0f3430d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95592a4a336649a9a859836daaf77ddf'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:19.145114 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-d7e7d0f8-a5d4-4e4d-8cd1-aafdc0f3430d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95592a4a336649a9a859836daaf77ddf'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:19.229763 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-d7e7d0f8-a5d4-4e4d-8cd1-aafdc0f3430d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95592a4a336649a9a859836daaf77ddf'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:19.246448 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1273/2503] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:32:18 2022] DELETE /identity/v3/users/95592a4a336649a9a859836daaf77ddf => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:32:19.353687 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-a7e3799b-b277-4f56-a575-be6eb916cab8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b71b044d7bd41a983930004ad8a1751'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:19.365912 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-a7e3799b-b277-4f56-a575-be6eb916cab8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b71b044d7bd41a983930004ad8a1751'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:19.366646 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-a7e3799b-b277-4f56-a575-be6eb916cab8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b71b044d7bd41a983930004ad8a1751'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:19.373418 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-a7e3799b-b277-4f56-a575-be6eb916cab8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b71b044d7bd41a983930004ad8a1751'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:19.374068 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-a7e3799b-b277-4f56-a575-be6eb916cab8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b71b044d7bd41a983930004ad8a1751'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:19.438361 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e7d0f8-a5d4-4e4d-8cd1-aafdc0f3430d admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:19.438791 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-d7e7d0f8-a5d4-4e4d-8cd1-aafdc0f3430d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:19.462193 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1231/2504] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:32:19 2022] DELETE /identity/v3/users/6b71b044d7bd41a983930004ad8a1751 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:32:19.469475 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b454d51c-a8f0-458e-8d7f-7b0793d1c4fa None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:19.469793 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b454d51c-a8f0-458e-8d7f-7b0793d1c4fa None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:19.470064 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b454d51c-a8f0-458e-8d7f-7b0793d1c4fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:19.501949 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-b454d51c-a8f0-458e-8d7f-7b0793d1c4fa None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:32:19.504295 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-b454d51c-a8f0-458e-8d7f-7b0793d1c4fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:19.545007 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e7d0f8-a5d4-4e4d-8cd1-aafdc0f3430d admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:19.593210 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1232/2505] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:32:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:32:19.600082 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b454d51c-a8f0-458e-8d7f-7b0793d1c4fa None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:19.601883 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-b454d51c-a8f0-458e-8d7f-7b0793d1c4fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:19.608146 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:19.633972 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e757396-be6d-418b-8ce2-83107ae07b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:19.634878 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e757396-be6d-418b-8ce2-83107ae07b01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:19.635286 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e757396-be6d-418b-8ce2-83107ae07b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:19.635726 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e757396-be6d-418b-8ce2-83107ae07b01 admin admin] PATH_INFO: `/v3/users/4087b29e666c4afaa36118e82eb169d3` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:19.637345 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e757396-be6d-418b-8ce2-83107ae07b01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4087b29e666c4afaa36118e82eb169d3)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:19.639702 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e757396-be6d-418b-8ce2-83107ae07b01 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:19.670957 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b454d51c-a8f0-458e-8d7f-7b0793d1c4fa None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:19.676681 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0657d9b-2bde-47cf-a957-c4ec3fb3077d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:19.677337 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0657d9b-2bde-47cf-a957-c4ec3fb3077d service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:19.677616 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0657d9b-2bde-47cf-a957-c4ec3fb3077d service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:19.677886 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0657d9b-2bde-47cf-a957-c4ec3fb3077d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:19.687161 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0657d9b-2bde-47cf-a957-c4ec3fb3077d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:19.688796 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0657d9b-2bde-47cf-a957-c4ec3fb3077d service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:19.698416 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1233/2506] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:32:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:32:19.808115 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0657d9b-2bde-47cf-a957-c4ec3fb3077d service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:19.809550 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-b0657d9b-2bde-47cf-a957-c4ec3fb3077d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:19.831300 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0657d9b-2bde-47cf-a957-c4ec3fb3077d service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:19.845377 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9244faf9-1da7-4157-80c0-c7f436afb866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:19.846101 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9244faf9-1da7-4157-80c0-c7f436afb866 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:19.846380 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9244faf9-1da7-4157-80c0-c7f436afb866 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:19.846677 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9244faf9-1da7-4157-80c0-c7f436afb866 admin admin] PATH_INFO: `/v3/projects/d28d7ee4c45747f685386ad235e48ff4` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:19.848360 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9244faf9-1da7-4157-80c0-c7f436afb866 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d28d7ee4c45747f685386ad235e48ff4)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:19.850453 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9244faf9-1da7-4157-80c0-c7f436afb866 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:20.036730 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-6e757396-be6d-418b-8ce2-83107ae07b01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4087b29e666c4afaa36118e82eb169d3'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:20.048468 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-6e757396-be6d-418b-8ce2-83107ae07b01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4087b29e666c4afaa36118e82eb169d3'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:20.049216 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-6e757396-be6d-418b-8ce2-83107ae07b01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4087b29e666c4afaa36118e82eb169d3'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:20.049928 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-6e757396-be6d-418b-8ce2-83107ae07b01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4087b29e666c4afaa36118e82eb169d3'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:20.182883 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-9244faf9-1da7-4157-80c0-c7f436afb866 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd28d7ee4c45747f685386ad235e48ff4', 'request_id': 'req-9244faf9-1da7-4157-80c0-c7f436afb866'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:20.190453 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-9244faf9-1da7-4157-80c0-c7f436afb866 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd28d7ee4c45747f685386ad235e48ff4', 'request_id': 'req-9244faf9-1da7-4157-80c0-c7f436afb866'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:20.206196 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-6e757396-be6d-418b-8ce2-83107ae07b01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4087b29e666c4afaa36118e82eb169d3'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:20.230631 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1274/2507] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:32:19 2022] DELETE /identity/v3/users/4087b29e666c4afaa36118e82eb169d3 => generated 0 bytes in 795 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:32:20.311104 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-9244faf9-1da7-4157-80c0-c7f436afb866 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd28d7ee4c45747f685386ad235e48ff4', 'request_id': 'req-9244faf9-1da7-4157-80c0-c7f436afb866'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:20.325539 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-9244faf9-1da7-4157-80c0-c7f436afb866 admin admin] The token cache is being invalidate because project d28d7ee4c45747f685386ad235e48ff4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:32:20.325860 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-9244faf9-1da7-4157-80c0-c7f436afb866 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:20.326570 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-9244faf9-1da7-4157-80c0-c7f436afb866 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:20.332035 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1234/2508] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:32:19 2022] DELETE /identity/v3/projects/d28d7ee4c45747f685386ad235e48ff4 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:32:20.334630 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25599aa3-b0a4-40c0-8dfd-2a3df5c31bce None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:20.335260 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25599aa3-b0a4-40c0-8dfd-2a3df5c31bce None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:20.335602 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25599aa3-b0a4-40c0-8dfd-2a3df5c31bce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:20.359018 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-25599aa3-b0a4-40c0-8dfd-2a3df5c31bce None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:32:20.360329 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-25599aa3-b0a4-40c0-8dfd-2a3df5c31bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:20.515551 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1275/2509] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:32:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:32:20.527605 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6d7e13-8d0e-4a5c-b9bd-6f3fa6aa5573 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:20.528513 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6d7e13-8d0e-4a5c-b9bd-6f3fa6aa5573 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:20.528787 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6d7e13-8d0e-4a5c-b9bd-6f3fa6aa5573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:20.555912 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-cd6d7e13-8d0e-4a5c-b9bd-6f3fa6aa5573 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:32:20.557153 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-cd6d7e13-8d0e-4a5c-b9bd-6f3fa6aa5573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:20.566854 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1235/2510] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:32:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:32:20.572437 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25599aa3-b0a4-40c0-8dfd-2a3df5c31bce None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:20.573495 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6d7e13-8d0e-4a5c-b9bd-6f3fa6aa5573 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:20.576762 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-cd6d7e13-8d0e-4a5c-b9bd-6f3fa6aa5573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:20.592591 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6d7e13-8d0e-4a5c-b9bd-6f3fa6aa5573 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:20.605131 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25599aa3-b0a4-40c0-8dfd-2a3df5c31bce None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:20.608369 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1906ef73-38b0-4672-9d12-99a97838152e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:20.610672 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1906ef73-38b0-4672-9d12-99a97838152e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:20.611560 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1906ef73-38b0-4672-9d12-99a97838152e admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:20.626914 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1906ef73-38b0-4672-9d12-99a97838152e admin admin] PATH_INFO: `/v3/projects/301dd99bc8f048f7abfda53b2f5fbd67` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:20.627081 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea23766-fe24-45ed-8227-148d25bc6302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:20.627081 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea23766-fe24-45ed-8227-148d25bc6302 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:20.627081 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea23766-fe24-45ed-8227-148d25bc6302 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:20.627520 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1906ef73-38b0-4672-9d12-99a97838152e admin admin] RBAC: Authorizing `identity:delete_project(project_id=301dd99bc8f048f7abfda53b2f5fbd67)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:20.627624 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea23766-fe24-45ed-8227-148d25bc6302 admin admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:20.627624 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea23766-fe24-45ed-8227-148d25bc6302 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:32:20.627624 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea23766-fe24-45ed-8227-148d25bc6302 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:20.627894 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1906ef73-38b0-4672-9d12-99a97838152e admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:20.627992 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea23766-fe24-45ed-8227-148d25bc6302 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:20.637419 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1276/2511] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Aug 4 21:32:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:32:20.641816 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea23766-fe24-45ed-8227-148d25bc6302 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:20.646197 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-bea23766-fe24-45ed-8227-148d25bc6302 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:20.667003 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea23766-fe24-45ed-8227-148d25bc6302 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:20.679240 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a0d1c8-75c4-4582-a020-c35267516be8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:20.679944 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a0d1c8-75c4-4582-a020-c35267516be8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:20.680225 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a0d1c8-75c4-4582-a020-c35267516be8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:20.680500 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a0d1c8-75c4-4582-a020-c35267516be8 admin admin] PATH_INFO: `/v3/projects/48234ebc0b1341719ebf2a9419cb6b54` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:20.682173 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a0d1c8-75c4-4582-a020-c35267516be8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48234ebc0b1341719ebf2a9419cb6b54)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:20.684326 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a0d1c8-75c4-4582-a020-c35267516be8 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:20.759606 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-1906ef73-38b0-4672-9d12-99a97838152e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '301dd99bc8f048f7abfda53b2f5fbd67', 'request_id': 'req-1906ef73-38b0-4672-9d12-99a97838152e'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:20.765457 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-1906ef73-38b0-4672-9d12-99a97838152e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '301dd99bc8f048f7abfda53b2f5fbd67', 'request_id': 'req-1906ef73-38b0-4672-9d12-99a97838152e'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:20.839384 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-67a0d1c8-75c4-4582-a020-c35267516be8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48234ebc0b1341719ebf2a9419cb6b54', 'request_id': 'req-67a0d1c8-75c4-4582-a020-c35267516be8'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:20.844458 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-1906ef73-38b0-4672-9d12-99a97838152e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '301dd99bc8f048f7abfda53b2f5fbd67', 'request_id': 'req-1906ef73-38b0-4672-9d12-99a97838152e'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:20.849319 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-67a0d1c8-75c4-4582-a020-c35267516be8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48234ebc0b1341719ebf2a9419cb6b54', 'request_id': 'req-67a0d1c8-75c4-4582-a020-c35267516be8'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:20.862129 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-1906ef73-38b0-4672-9d12-99a97838152e admin admin] The token cache is being invalidate because project 301dd99bc8f048f7abfda53b2f5fbd67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:32:20.862747 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-1906ef73-38b0-4672-9d12-99a97838152e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:20.863614 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-1906ef73-38b0-4672-9d12-99a97838152e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:20.865341 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1236/2512] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:32:20 2022] DELETE /identity/v3/projects/301dd99bc8f048f7abfda53b2f5fbd67 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:32:20.866347 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1906ef73-38b0-4672-9d12-99a97838152e admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:20.868076 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-1906ef73-38b0-4672-9d12-99a97838152e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:20.944551 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1906ef73-38b0-4672-9d12-99a97838152e admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:20.953163 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-67a0d1c8-75c4-4582-a020-c35267516be8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48234ebc0b1341719ebf2a9419cb6b54', 'request_id': 'req-67a0d1c8-75c4-4582-a020-c35267516be8'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:20.968296 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-67a0d1c8-75c4-4582-a020-c35267516be8 admin admin] The token cache is being invalidate because project 48234ebc0b1341719ebf2a9419cb6b54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:32:20.969453 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-67a0d1c8-75c4-4582-a020-c35267516be8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:20.970488 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-67a0d1c8-75c4-4582-a020-c35267516be8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:20.972267 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1277/2513] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:32:20 2022] DELETE /identity/v3/projects/48234ebc0b1341719ebf2a9419cb6b54 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:32:20.973575 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a0d1c8-75c4-4582-a020-c35267516be8 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:20.975920 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-67a0d1c8-75c4-4582-a020-c35267516be8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:21.016760 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37d544b-2ff6-4d5a-8272-ee38745620e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:21.017692 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37d544b-2ff6-4d5a-8272-ee38745620e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:21.018141 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37d544b-2ff6-4d5a-8272-ee38745620e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:21.018614 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37d544b-2ff6-4d5a-8272-ee38745620e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:21.019803 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37d544b-2ff6-4d5a-8272-ee38745620e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:21.022596 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37d544b-2ff6-4d5a-8272-ee38745620e9 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:21.078459 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a0d1c8-75c4-4582-a020-c35267516be8 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:21.084219 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd9a523-2ded-47e0-995c-d868c7d0d3ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:21.085176 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd9a523-2ded-47e0-995c-d868c7d0d3ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:21.085469 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd9a523-2ded-47e0-995c-d868c7d0d3ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:21.085744 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd9a523-2ded-47e0-995c-d868c7d0d3ba admin admin] PATH_INFO: `/v3/users/4bd9b51d4bd6488999b96fd1042efd46` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:21.087836 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd9a523-2ded-47e0-995c-d868c7d0d3ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=4bd9b51d4bd6488999b96fd1042efd46)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:21.090243 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd9a523-2ded-47e0-995c-d868c7d0d3ba admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:21.208397 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1237/2514] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:32:20 2022] POST /identity/v3/projects => generated 365 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:32:21.210116 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e5cf2-4535-416a-9100-ab488597193e None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:21.210556 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e5cf2-4535-416a-9100-ab488597193e None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:21.210979 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e5cf2-4535-416a-9100-ab488597193e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:21.248091 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-fd9e5cf2-4535-416a-9100-ab488597193e None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:32:21.249161 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-fd9e5cf2-4535-416a-9100-ab488597193e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:21.325013 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1238/2515] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:32:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:32:21.326386 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9e5cf2-4535-416a-9100-ab488597193e None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:21.327478 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-fd9e5cf2-4535-416a-9100-ab488597193e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:21.340148 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9e5cf2-4535-416a-9100-ab488597193e None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:21.352371 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6d2128-6bcf-4c05-b9d2-2ddaefa569cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:21.355795 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6d2128-6bcf-4c05-b9d2-2ddaefa569cb admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:21.355795 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6d2128-6bcf-4c05-b9d2-2ddaefa569cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:21.355795 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6d2128-6bcf-4c05-b9d2-2ddaefa569cb admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:21.356272 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6d2128-6bcf-4c05-b9d2-2ddaefa569cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:21.358244 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6d2128-6bcf-4c05-b9d2-2ddaefa569cb admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:21.366768 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-ccd9a523-2ded-47e0-995c-d868c7d0d3ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4bd9b51d4bd6488999b96fd1042efd46'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:21.382102 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-ccd9a523-2ded-47e0-995c-d868c7d0d3ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4bd9b51d4bd6488999b96fd1042efd46'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:21.384063 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-ccd9a523-2ded-47e0-995c-d868c7d0d3ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4bd9b51d4bd6488999b96fd1042efd46'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:21.384063 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-ccd9a523-2ded-47e0-995c-d868c7d0d3ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4bd9b51d4bd6488999b96fd1042efd46'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:21.435737 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1239/2516] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:32:21 2022] POST /identity/v3/users => generated 283 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:32:21.436122 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001c7d03-71b8-4a21-a7a1-2cc283c31f59 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:21.436369 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001c7d03-71b8-4a21-a7a1-2cc283c31f59 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:21.436874 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001c7d03-71b8-4a21-a7a1-2cc283c31f59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:21.447927 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-ccd9a523-2ded-47e0-995c-d868c7d0d3ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4bd9b51d4bd6488999b96fd1042efd46'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:21.463375 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-001c7d03-71b8-4a21-a7a1-2cc283c31f59 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:32:21.464744 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-001c7d03-71b8-4a21-a7a1-2cc283c31f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:21.466007 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1278/2517] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:32:20 2022] DELETE /identity/v3/users/4bd9b51d4bd6488999b96fd1042efd46 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:32:21.466689 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd9a523-2ded-47e0-995c-d868c7d0d3ba admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:21.468045 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-ccd9a523-2ded-47e0-995c-d868c7d0d3ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:21.535532 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1240/2518] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:32:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:32:21.548245 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd9a523-2ded-47e0-995c-d868c7d0d3ba admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:21.551470 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001c7d03-71b8-4a21-a7a1-2cc283c31f59 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:21.554785 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cc8517-b86a-4ee5-8c36-74c11bedf01f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:21.557371 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cc8517-b86a-4ee5-8c36-74c11bedf01f admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:21.557371 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cc8517-b86a-4ee5-8c36-74c11bedf01f admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:21.557371 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cc8517-b86a-4ee5-8c36-74c11bedf01f admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:21.558243 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cc8517-b86a-4ee5-8c36-74c11bedf01f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:21.560312 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cc8517-b86a-4ee5-8c36-74c11bedf01f admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:21.573609 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001c7d03-71b8-4a21-a7a1-2cc283c31f59 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:21.582349 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1279/2519] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:32:21 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:32:21.584315 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cc8517-b86a-4ee5-8c36-74c11bedf01f admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:21.586478 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c518b017-3e35-4708-8590-6a4b4ebd3f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:21.587385 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c518b017-3e35-4708-8590-6a4b4ebd3f83 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:21.587843 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c518b017-3e35-4708-8590-6a4b4ebd3f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:21.588280 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c518b017-3e35-4708-8590-6a4b4ebd3f83 admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:21.589161 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c518b017-3e35-4708-8590-6a4b4ebd3f83 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:32:21.589632 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c518b017-3e35-4708-8590-6a4b4ebd3f83 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:21.591519 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c518b017-3e35-4708-8590-6a4b4ebd3f83 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:21.603427 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cc8517-b86a-4ee5-8c36-74c11bedf01f admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:21.615883 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0250fe-2445-409f-a5c2-f0ac2ab26731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:21.617940 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0250fe-2445-409f-a5c2-f0ac2ab26731 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:21.618304 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0250fe-2445-409f-a5c2-f0ac2ab26731 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:21.618653 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0250fe-2445-409f-a5c2-f0ac2ab26731 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:21.619536 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0250fe-2445-409f-a5c2-f0ac2ab26731 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:21.621566 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0250fe-2445-409f-a5c2-f0ac2ab26731 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:21.630300 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1241/2520] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Aug 4 21:32:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:32:21.631531 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c518b017-3e35-4708-8590-6a4b4ebd3f83 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:21.641974 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1280/2521] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:32:21 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:32:21.643499 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c518b017-3e35-4708-8590-6a4b4ebd3f83 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:21.646090 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0250fe-2445-409f-a5c2-f0ac2ab26731 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:21.649377 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8d193f-c4f8-4bc0-aef8-9f58f6969483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:21.650013 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8d193f-c4f8-4bc0-aef8-9f58f6969483 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:21.650296 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8d193f-c4f8-4bc0-aef8-9f58f6969483 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:21.650639 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8d193f-c4f8-4bc0-aef8-9f58f6969483 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:21.651860 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8d193f-c4f8-4bc0-aef8-9f58f6969483 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:21.654244 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8d193f-c4f8-4bc0-aef8-9f58f6969483 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:21.665526 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0250fe-2445-409f-a5c2-f0ac2ab26731 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:21.671690 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d533a511-910f-424f-afca-a66d69584a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:21.673118 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d533a511-910f-424f-afca-a66d69584a30 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:21.673641 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d533a511-910f-424f-afca-a66d69584a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:21.674018 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d533a511-910f-424f-afca-a66d69584a30 admin admin] PATH_INFO: `/v3/projects/4a4d68ac121e4a61a9f7819f05c78d56/users/afb40c7e64814d3aa37decc43470a4dd/roles/8feebc60363c4feb870718b1721e5324` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:21.700013 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d533a511-910f-424f-afca-a66d69584a30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a4d68ac121e4a61a9f7819f05c78d56, user_id=afb40c7e64814d3aa37decc43470a4dd, role_id=8feebc60363c4feb870718b1721e5324)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:21.702917 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d533a511-910f-424f-afca-a66d69584a30 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:21.789253 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1242/2522] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:32:21 2022] POST /identity/v3/projects => generated 403 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:32:21.790930 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8d193f-c4f8-4bc0-aef8-9f58f6969483 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:21.808951 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1281/2523] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:32:21 2022] PUT /identity/v3/projects/4a4d68ac121e4a61a9f7819f05c78d56/users/afb40c7e64814d3aa37decc43470a4dd/roles/8feebc60363c4feb870718b1721e5324 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:32:21.824327 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de91554b-3735-4b61-9e17-833468b72e59 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:21.826636 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de91554b-3735-4b61-9e17-833468b72e59 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:21.826988 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de91554b-3735-4b61-9e17-833468b72e59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:21.872597 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:21.888035 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-de91554b-3735-4b61-9e17-833468b72e59 None None] MFA Rules not processed for user `afb40c7e64814d3aa37decc43470a4dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:32:21.889596 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-de91554b-3735-4b61-9e17-833468b72e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:21.948280 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8d193f-c4f8-4bc0-aef8-9f58f6969483 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:21.992272 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:22.030240 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1282/2524] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:32:21 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:32:22.039221 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de91554b-3735-4b61-9e17-833468b72e59 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:22.130865 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee43dc07-872d-4b47-a97a-1ce746749bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:22.131668 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee43dc07-872d-4b47-a97a-1ce746749bfe admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:22.132161 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee43dc07-872d-4b47-a97a-1ce746749bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:22.132741 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee43dc07-872d-4b47-a97a-1ce746749bfe admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:22.133942 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee43dc07-872d-4b47-a97a-1ce746749bfe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:22.137905 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee43dc07-872d-4b47-a97a-1ce746749bfe admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:22.143685 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de91554b-3735-4b61-9e17-833468b72e59 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:22.149430 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e478ebe-c680-4c2a-945c-6fb2b33ff9b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:22.151666 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e478ebe-c680-4c2a-945c-6fb2b33ff9b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:22.152193 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e478ebe-c680-4c2a-945c-6fb2b33ff9b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:22.152825 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e478ebe-c680-4c2a-945c-6fb2b33ff9b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:22.153622 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e478ebe-c680-4c2a-945c-6fb2b33ff9b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:22.166645 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e478ebe-c680-4c2a-945c-6fb2b33ff9b3 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:22.299140 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1243/2525] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:32:21 2022] POST /identity/v3/users => generated 302 bytes in 501 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:32:22.299140 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee43dc07-872d-4b47-a97a-1ce746749bfe admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:22.299140 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-ee43dc07-872d-4b47-a97a-1ce746749bfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:22.313774 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1283/2526] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:32:22 2022] POST /identity/v3/projects => generated 363 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:32:22.315650 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e478ebe-c680-4c2a-945c-6fb2b33ff9b3 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:22.330382 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee43dc07-872d-4b47-a97a-1ce746749bfe admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:22.409017 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1065b65-dee8-4539-95fd-bd4b36ff2fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:22.409958 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1065b65-dee8-4539-95fd-bd4b36ff2fd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:22.410411 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1065b65-dee8-4539-95fd-bd4b36ff2fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:22.410861 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1065b65-dee8-4539-95fd-bd4b36ff2fd3 admin admin] PATH_INFO: `/v3/projects/e8e42b51e53b49a68f903fab6aa5d917` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:22.412399 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1065b65-dee8-4539-95fd-bd4b36ff2fd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8e42b51e53b49a68f903fab6aa5d917)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:22.414572 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1065b65-dee8-4539-95fd-bd4b36ff2fd3 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:22.446550 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e478ebe-c680-4c2a-945c-6fb2b33ff9b3 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:22.451442 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffb04bb-54f4-477f-a867-9807eb093a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:22.455929 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffb04bb-54f4-477f-a867-9807eb093a6b admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:22.455929 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffb04bb-54f4-477f-a867-9807eb093a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:22.455929 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffb04bb-54f4-477f-a867-9807eb093a6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:22.455929 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffb04bb-54f4-477f-a867-9807eb093a6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:22.455929 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffb04bb-54f4-477f-a867-9807eb093a6b admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:22.467429 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1284/2527] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:32:22 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:32:22.467429 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffb04bb-54f4-477f-a867-9807eb093a6b admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:22.485439 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffb04bb-54f4-477f-a867-9807eb093a6b admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:22.490937 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff38049-e349-47ae-83cd-327a5b51e5c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:22.491705 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff38049-e349-47ae-83cd-327a5b51e5c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:22.491988 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff38049-e349-47ae-83cd-327a5b51e5c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:22.492263 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff38049-e349-47ae-83cd-327a5b51e5c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:22.492945 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff38049-e349-47ae-83cd-327a5b51e5c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:22.495843 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff38049-e349-47ae-83cd-327a5b51e5c0 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:22.557182 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-e1065b65-dee8-4539-95fd-bd4b36ff2fd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8e42b51e53b49a68f903fab6aa5d917', 'request_id': 'req-e1065b65-dee8-4539-95fd-bd4b36ff2fd3'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:22.563829 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-e1065b65-dee8-4539-95fd-bd4b36ff2fd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8e42b51e53b49a68f903fab6aa5d917', 'request_id': 'req-e1065b65-dee8-4539-95fd-bd4b36ff2fd3'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:22.630430 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1285/2528] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:32:22 2022] POST /identity/v3/users => generated 288 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:32:22.631559 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff38049-e349-47ae-83cd-327a5b51e5c0 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:22.642363 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff38049-e349-47ae-83cd-327a5b51e5c0 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:22.644454 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-e1065b65-dee8-4539-95fd-bd4b36ff2fd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8e42b51e53b49a68f903fab6aa5d917', 'request_id': 'req-e1065b65-dee8-4539-95fd-bd4b36ff2fd3'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:22.665833 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-e1065b65-dee8-4539-95fd-bd4b36ff2fd3 admin admin] The token cache is being invalidate because project e8e42b51e53b49a68f903fab6aa5d917 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:32:22.666439 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-e1065b65-dee8-4539-95fd-bd4b36ff2fd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:22.667386 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-e1065b65-dee8-4539-95fd-bd4b36ff2fd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:22.673407 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1244/2529] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:32:22 2022] DELETE /identity/v3/projects/e8e42b51e53b49a68f903fab6aa5d917 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:32:22.687750 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1065b65-dee8-4539-95fd-bd4b36ff2fd3 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:22.687750 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-e1065b65-dee8-4539-95fd-bd4b36ff2fd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:22.783828 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088420a7-94e5-4e4a-8d11-0736711645dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:22.785917 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088420a7-94e5-4e4a-8d11-0736711645dd admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:22.786223 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088420a7-94e5-4e4a-8d11-0736711645dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:22.786504 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088420a7-94e5-4e4a-8d11-0736711645dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:22.787255 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088420a7-94e5-4e4a-8d11-0736711645dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:22.790666 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088420a7-94e5-4e4a-8d11-0736711645dd admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:22.815981 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1286/2530] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:32:22 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:32:22.815981 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e4c1c6-1abc-4668-94d6-aea0eecf393b None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:22.815981 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e4c1c6-1abc-4668-94d6-aea0eecf393b None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:22.820818 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e4c1c6-1abc-4668-94d6-aea0eecf393b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:22.840744 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1065b65-dee8-4539-95fd-bd4b36ff2fd3 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:22.863144 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79df55e-3e89-4256-85fe-ab659a8a4c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:22.863824 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79df55e-3e89-4256-85fe-ab659a8a4c16 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:22.864110 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79df55e-3e89-4256-85fe-ab659a8a4c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:22.864391 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79df55e-3e89-4256-85fe-ab659a8a4c16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:22.865205 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79df55e-3e89-4256-85fe-ab659a8a4c16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:22.866869 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79df55e-3e89-4256-85fe-ab659a8a4c16 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:22.878086 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-a2e4c1c6-1abc-4668-94d6-aea0eecf393b None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:32:22.878971 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-a2e4c1c6-1abc-4668-94d6-aea0eecf393b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:22.888722 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1245/2531] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:32:22 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:32:22.889589 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79df55e-3e89-4256-85fe-ab659a8a4c16 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:22.897817 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-a79df55e-3e89-4256-85fe-ab659a8a4c16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:22.908098 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1287/2532] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:32:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:32:22.908098 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e4c1c6-1abc-4668-94d6-aea0eecf393b None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:22.919426 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e4c1c6-1abc-4668-94d6-aea0eecf393b None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:22.924869 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79df55e-3e89-4256-85fe-ab659a8a4c16 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:22.930012 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417b4b5f-b001-4649-a0f4-69019bb81940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:22.930709 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417b4b5f-b001-4649-a0f4-69019bb81940 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:22.930996 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417b4b5f-b001-4649-a0f4-69019bb81940 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:22.931602 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417b4b5f-b001-4649-a0f4-69019bb81940 admin admin] PATH_INFO: `/v3/projects/405f9d7d02c642fc822ffef7959ab0ca/users/f765a959908e4992ad41ec66d21577f7/roles/a432dd4735944ecfab2fa5ce1705d356` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:22.951521 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930b8a32-399a-4a96-96b2-f8fae864dc1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:22.952232 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930b8a32-399a-4a96-96b2-f8fae864dc1b admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:22.952532 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930b8a32-399a-4a96-96b2-f8fae864dc1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:22.953583 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930b8a32-399a-4a96-96b2-f8fae864dc1b admin admin] PATH_INFO: `/v3/projects/750fe45bcd0649759bdbcde0c683acab/users/d5a5475497114d73913ef394d1f8d9a6/roles/8feebc60363c4feb870718b1721e5324` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:22.967895 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417b4b5f-b001-4649-a0f4-69019bb81940 admin admin] RBAC: Authorizing `identity:create_grant(project_id=405f9d7d02c642fc822ffef7959ab0ca, user_id=f765a959908e4992ad41ec66d21577f7, role_id=a432dd4735944ecfab2fa5ce1705d356)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:22.972164 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417b4b5f-b001-4649-a0f4-69019bb81940 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:22.974751 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:22.989778 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930b8a32-399a-4a96-96b2-f8fae864dc1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=750fe45bcd0649759bdbcde0c683acab, user_id=d5a5475497114d73913ef394d1f8d9a6, role_id=8feebc60363c4feb870718b1721e5324)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:22.993110 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930b8a32-399a-4a96-96b2-f8fae864dc1b admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:23.108182 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1288/2533] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:32:22 2022] PUT /identity/v3/projects/405f9d7d02c642fc822ffef7959ab0ca/users/f765a959908e4992ad41ec66d21577f7/roles/a432dd4735944ecfab2fa5ce1705d356 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:32:23.112613 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79046fca-dd6d-48ba-80b7-b5030079c6b0 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:23.113000 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79046fca-dd6d-48ba-80b7-b5030079c6b0 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:23.113175 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79046fca-dd6d-48ba-80b7-b5030079c6b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:23.152409 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1246/2534] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:32:22 2022] PUT /identity/v3/projects/750fe45bcd0649759bdbcde0c683acab/users/d5a5475497114d73913ef394d1f8d9a6/roles/8feebc60363c4feb870718b1721e5324 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:32:23.153927 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930b8a32-399a-4a96-96b2-f8fae864dc1b admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:23.165136 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-79046fca-dd6d-48ba-80b7-b5030079c6b0 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:32:23.168677 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-79046fca-dd6d-48ba-80b7-b5030079c6b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:23.327008 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1289/2535] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:32:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:32:23.330838 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6880535c-02fe-4149-b167-30fd54b0a4ed None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:23.330838 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6880535c-02fe-4149-b167-30fd54b0a4ed None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:23.330838 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6880535c-02fe-4149-b167-30fd54b0a4ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:23.346942 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930b8a32-399a-4a96-96b2-f8fae864dc1b admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:23.363387 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dbce60-5a15-4976-a47f-48d9e800b019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:23.364523 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dbce60-5a15-4976-a47f-48d9e800b019 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:23.365045 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dbce60-5a15-4976-a47f-48d9e800b019 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:23.365537 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dbce60-5a15-4976-a47f-48d9e800b019 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:23.366502 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dbce60-5a15-4976-a47f-48d9e800b019 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:23.370602 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dbce60-5a15-4976-a47f-48d9e800b019 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:23.406843 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-6880535c-02fe-4149-b167-30fd54b0a4ed None None] MFA Rules not processed for user `d5a5475497114d73913ef394d1f8d9a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:32:23.407059 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1247/2536] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:32:23 2022] GET /identity/v3/roles => generated 1420 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:32:23.407059 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dbce60-5a15-4976-a47f-48d9e800b019 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:23.407262 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-6880535c-02fe-4149-b167-30fd54b0a4ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:23.421050 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dbce60-5a15-4976-a47f-48d9e800b019 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:23.448120 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d958ad-7a89-49fb-9399-f666983d4fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:23.448966 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d958ad-7a89-49fb-9399-f666983d4fda admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:23.449621 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d958ad-7a89-49fb-9399-f666983d4fda admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:23.450005 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d958ad-7a89-49fb-9399-f666983d4fda admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:23.450818 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d958ad-7a89-49fb-9399-f666983d4fda admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:32:23.451230 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d958ad-7a89-49fb-9399-f666983d4fda admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:23.453925 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d958ad-7a89-49fb-9399-f666983d4fda admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:23.476488 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:23.481083 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1248/2537] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Aug 4 21:32:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:32:23.485713 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d958ad-7a89-49fb-9399-f666983d4fda admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:23.504966 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d958ad-7a89-49fb-9399-f666983d4fda admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:23.520481 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a479d9-a349-4033-8f66-a08c7c7610cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:23.521198 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a479d9-a349-4033-8f66-a08c7c7610cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:23.521476 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a479d9-a349-4033-8f66-a08c7c7610cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:23.521758 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a479d9-a349-4033-8f66-a08c7c7610cc admin admin] PATH_INFO: `/v3/projects/405f9d7d02c642fc822ffef7959ab0ca/users/f765a959908e4992ad41ec66d21577f7/roles/a432dd4735944ecfab2fa5ce1705d356` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:23.524720 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a479d9-a349-4033-8f66-a08c7c7610cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=405f9d7d02c642fc822ffef7959ab0ca, user_id=f765a959908e4992ad41ec66d21577f7, role_id=a432dd4735944ecfab2fa5ce1705d356)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:23.528538 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a479d9-a349-4033-8f66-a08c7c7610cc admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:23.561485 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1249/2538] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:32:23 2022] PUT /identity/v3/projects/405f9d7d02c642fc822ffef7959ab0ca/users/f765a959908e4992ad41ec66d21577f7/roles/a432dd4735944ecfab2fa5ce1705d356 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:32:23.561485 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a479d9-a349-4033-8f66-a08c7c7610cc admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:23.572539 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1290/2539] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:32:23 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:32:23.573856 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f827d92-dbaf-4774-a7f8-33c5ed98abdf None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:23.574147 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f827d92-dbaf-4774-a7f8-33c5ed98abdf None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:23.574440 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f827d92-dbaf-4774-a7f8-33c5ed98abdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:23.645376 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-0f827d92-dbaf-4774-a7f8-33c5ed98abdf None None] MFA Rules not processed for user `f765a959908e4992ad41ec66d21577f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:32:23.646628 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-0f827d92-dbaf-4774-a7f8-33c5ed98abdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:23.769808 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1291/2540] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:32:23 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:32:23.770984 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f827d92-dbaf-4774-a7f8-33c5ed98abdf None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:23.778796 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a479d9-a349-4033-8f66-a08c7c7610cc admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:23.792496 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f827d92-dbaf-4774-a7f8-33c5ed98abdf None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:23.792833 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a304e7d-0607-4a8e-8bec-5562b2a5d133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:23.798053 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a304e7d-0607-4a8e-8bec-5562b2a5d133 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:23.798472 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a304e7d-0607-4a8e-8bec-5562b2a5d133 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:23.798650 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a304e7d-0607-4a8e-8bec-5562b2a5d133 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:23.799368 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a304e7d-0607-4a8e-8bec-5562b2a5d133 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:23.800660 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2351d13-fd01-4c0e-af7f-e9e65e0e52ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:23.801634 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2351d13-fd01-4c0e-af7f-e9e65e0e52ce admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:23.802158 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2351d13-fd01-4c0e-af7f-e9e65e0e52ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:23.802655 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2351d13-fd01-4c0e-af7f-e9e65e0e52ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:23.803659 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2351d13-fd01-4c0e-af7f-e9e65e0e52ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:23.804857 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a304e7d-0607-4a8e-8bec-5562b2a5d133 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:23.806185 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2351d13-fd01-4c0e-af7f-e9e65e0e52ce admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:23.910452 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1292/2541] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:32:23 2022] POST /identity/v3/projects => generated 405 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:32:23.910452 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2351d13-fd01-4c0e-af7f-e9e65e0e52ce admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:23.912449 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-f2351d13-fd01-4c0e-af7f-e9e65e0e52ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:23.942042 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1250/2542] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:32:23 2022] POST /identity/v3/projects => generated 409 bytes in 384 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:32:23.943034 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a304e7d-0607-4a8e-8bec-5562b2a5d133 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:24.030111 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2351d13-fd01-4c0e-af7f-e9e65e0e52ce admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:24.051276 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a304e7d-0607-4a8e-8bec-5562b2a5d133 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:24.062583 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784b3bf9-8a7b-4fae-8fcd-8b319769a753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:24.063779 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784b3bf9-8a7b-4fae-8fcd-8b319769a753 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:24.064510 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784b3bf9-8a7b-4fae-8fcd-8b319769a753 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:24.065010 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784b3bf9-8a7b-4fae-8fcd-8b319769a753 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:24.066973 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784b3bf9-8a7b-4fae-8fcd-8b319769a753 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:24.071302 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784b3bf9-8a7b-4fae-8fcd-8b319769a753 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:24.091476 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7205992-d42f-4e76-9530-2d1624a2e1cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e31f2728056b460fa474478eb85ca1d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:24.092206 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7205992-d42f-4e76-9530-2d1624a2e1cb service cinder] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:24.092493 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7205992-d42f-4e76-9530-2d1624a2e1cb service cinder] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:24.092768 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7205992-d42f-4e76-9530-2d1624a2e1cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:24.179997 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7205992-d42f-4e76-9530-2d1624a2e1cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:24.182430 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7205992-d42f-4e76-9530-2d1624a2e1cb service cinder] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:24.184048 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1251/2543] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:32:23 2022] POST /identity/v3/users => generated 309 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:32:24.185571 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784b3bf9-8a7b-4fae-8fcd-8b319769a753 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:24.205004 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784b3bf9-8a7b-4fae-8fcd-8b319769a753 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:24.205399 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1293/2544] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:32:23 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:32:24.206314 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7205992-d42f-4e76-9530-2d1624a2e1cb service cinder] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:24.215942 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f5fd83-1910-4db2-bf33-451e9fa06730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:24.217004 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f5fd83-1910-4db2-bf33-451e9fa06730 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:24.217487 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f5fd83-1910-4db2-bf33-451e9fa06730 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:24.222619 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f5fd83-1910-4db2-bf33-451e9fa06730 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:24.222619 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f5fd83-1910-4db2-bf33-451e9fa06730 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:24.223985 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f5fd83-1910-4db2-bf33-451e9fa06730 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:24.254638 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7205992-d42f-4e76-9530-2d1624a2e1cb service cinder] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:24.258936 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:24.271631 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60219407-3978-4013-9bd4-df42a523380f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:24.272731 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60219407-3978-4013-9bd4-df42a523380f admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:24.274294 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60219407-3978-4013-9bd4-df42a523380f admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:24.274294 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60219407-3978-4013-9bd4-df42a523380f admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:24.274294 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60219407-3978-4013-9bd4-df42a523380f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:24.276073 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60219407-3978-4013-9bd4-df42a523380f admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:24.290502 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1294/2545] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:32:24 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:32:24.291620 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60219407-3978-4013-9bd4-df42a523380f admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:24.302614 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60219407-3978-4013-9bd4-df42a523380f admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:24.309131 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837459d3-8b09-4847-b130-4faad782ea9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:24.309867 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837459d3-8b09-4847-b130-4faad782ea9b admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:24.310278 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837459d3-8b09-4847-b130-4faad782ea9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:24.310560 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837459d3-8b09-4847-b130-4faad782ea9b admin admin] PATH_INFO: `/v3/projects/5d8df6a1be084e44909e1e482298abaf/users/91dfaa1fdb224a059b6e4bb1aa4348e2/roles/a432dd4735944ecfab2fa5ce1705d356` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:24.327706 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837459d3-8b09-4847-b130-4faad782ea9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d8df6a1be084e44909e1e482298abaf, user_id=91dfaa1fdb224a059b6e4bb1aa4348e2, role_id=a432dd4735944ecfab2fa5ce1705d356)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:24.331351 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837459d3-8b09-4847-b130-4faad782ea9b admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:24.358443 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1252/2546] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:32:24 2022] POST /identity/v3/users => generated 305 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:32:24.360314 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f5fd83-1910-4db2-bf33-451e9fa06730 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:24.383002 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f5fd83-1910-4db2-bf33-451e9fa06730 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:24.394488 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9b30ce-e5df-411e-a5a9-437e03c65e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:24.395451 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9b30ce-e5df-411e-a5a9-437e03c65e03 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:24.395845 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9b30ce-e5df-411e-a5a9-437e03c65e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:24.396224 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9b30ce-e5df-411e-a5a9-437e03c65e03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:24.397229 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9b30ce-e5df-411e-a5a9-437e03c65e03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:24.399052 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9b30ce-e5df-411e-a5a9-437e03c65e03 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:24.410893 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1295/2547] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:32:24 2022] PUT /identity/v3/projects/5d8df6a1be084e44909e1e482298abaf/users/91dfaa1fdb224a059b6e4bb1aa4348e2/roles/a432dd4735944ecfab2fa5ce1705d356 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:32:24.416134 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837459d3-8b09-4847-b130-4faad782ea9b admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:24.430159 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1253/2548] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:32:24 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:32:24.431114 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9b30ce-e5df-411e-a5a9-437e03c65e03 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:24.448029 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9b30ce-e5df-411e-a5a9-437e03c65e03 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:24.526702 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837459d3-8b09-4847-b130-4faad782ea9b admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:24.532945 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7c7786-8a2e-4514-b652-ce6930139ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:24.533580 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7c7786-8a2e-4514-b652-ce6930139ff6 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:24.533862 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7c7786-8a2e-4514-b652-ce6930139ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:24.534146 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7c7786-8a2e-4514-b652-ce6930139ff6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:24.534848 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7c7786-8a2e-4514-b652-ce6930139ff6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:24.536605 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7c7786-8a2e-4514-b652-ce6930139ff6 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:24.543758 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e96b25-002b-46bb-89a8-de0e704d43bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:24.544542 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e96b25-002b-46bb-89a8-de0e704d43bc admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:24.544879 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e96b25-002b-46bb-89a8-de0e704d43bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:24.545255 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e96b25-002b-46bb-89a8-de0e704d43bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:24.546016 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e96b25-002b-46bb-89a8-de0e704d43bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:24.547438 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1296/2549] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:32:24 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:32:24.548109 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e96b25-002b-46bb-89a8-de0e704d43bc admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:24.548290 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7c7786-8a2e-4514-b652-ce6930139ff6 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:24.560643 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1254/2550] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:32:24 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:32:24.561581 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e96b25-002b-46bb-89a8-de0e704d43bc admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:24.573382 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e96b25-002b-46bb-89a8-de0e704d43bc admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:24.582070 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6b7e57-06b8-40a6-a720-16be2acd7bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:24.584087 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6b7e57-06b8-40a6-a720-16be2acd7bc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:24.584087 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6b7e57-06b8-40a6-a720-16be2acd7bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:24.584087 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6b7e57-06b8-40a6-a720-16be2acd7bc8 admin admin] PATH_INFO: `/v3/projects/5d8df6a1be084e44909e1e482298abaf/users/91dfaa1fdb224a059b6e4bb1aa4348e2/roles/a432dd4735944ecfab2fa5ce1705d356` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:24.586049 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6b7e57-06b8-40a6-a720-16be2acd7bc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d8df6a1be084e44909e1e482298abaf, user_id=91dfaa1fdb224a059b6e4bb1aa4348e2, role_id=a432dd4735944ecfab2fa5ce1705d356)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:24.588817 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6b7e57-06b8-40a6-a720-16be2acd7bc8 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:24.602349 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1255/2551] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:32:24 2022] PUT /identity/v3/projects/5d8df6a1be084e44909e1e482298abaf/users/91dfaa1fdb224a059b6e4bb1aa4348e2/roles/a432dd4735944ecfab2fa5ce1705d356 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:32:24.604316 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6b7e57-06b8-40a6-a720-16be2acd7bc8 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:24.632876 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7c7786-8a2e-4514-b652-ce6930139ff6 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:24.700118 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6b7e57-06b8-40a6-a720-16be2acd7bc8 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:24.711276 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7336a7fa-3228-4b29-9c21-55ae20627460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:24.715742 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7336a7fa-3228-4b29-9c21-55ae20627460 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:24.715742 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7336a7fa-3228-4b29-9c21-55ae20627460 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:24.715742 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7336a7fa-3228-4b29-9c21-55ae20627460 admin admin] PATH_INFO: `/v3/projects/3188dcfd3c944120bf5e34323cc4cc6d/users/f6ef1a44226c4ca183d189147cb78413/roles/8feebc60363c4feb870718b1721e5324` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:24.721078 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5a8af2-cce3-4190-ade1-55a629fe472f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e31f2728056b460fa474478eb85ca1d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:24.721708 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5a8af2-cce3-4190-ade1-55a629fe472f service cinder] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:24.721990 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5a8af2-cce3-4190-ade1-55a629fe472f service cinder] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:24.722306 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5a8af2-cce3-4190-ade1-55a629fe472f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:24.724311 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-2e5a8af2-cce3-4190-ade1-55a629fe472f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:24.745448 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7336a7fa-3228-4b29-9c21-55ae20627460 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3188dcfd3c944120bf5e34323cc4cc6d, user_id=f6ef1a44226c4ca183d189147cb78413, role_id=8feebc60363c4feb870718b1721e5324)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:24.748224 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7336a7fa-3228-4b29-9c21-55ae20627460 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:24.834809 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5a8af2-cce3-4190-ade1-55a629fe472f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:24.839081 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5a8af2-cce3-4190-ade1-55a629fe472f service cinder] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:24.839216 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1256/2552] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:32:24 2022] PUT /identity/v3/projects/3188dcfd3c944120bf5e34323cc4cc6d/users/f6ef1a44226c4ca183d189147cb78413/roles/8feebc60363c4feb870718b1721e5324 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:32:24.840659 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceec0cf-e691-4c51-abb2-4d479887b95e None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:24.841148 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceec0cf-e691-4c51-abb2-4d479887b95e None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:24.842169 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceec0cf-e691-4c51-abb2-4d479887b95e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:24.873071 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1297/2553] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:32:24 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:32:24.874504 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa82667-e3c8-4c87-8eb3-f81e4e53fdb6 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:24.875980 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa82667-e3c8-4c87-8eb3-f81e4e53fdb6 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:24.876766 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa82667-e3c8-4c87-8eb3-f81e4e53fdb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:24.892006 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-aceec0cf-e691-4c51-abb2-4d479887b95e None None] MFA Rules not processed for user `91dfaa1fdb224a059b6e4bb1aa4348e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:32:24.893679 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-aceec0cf-e691-4c51-abb2-4d479887b95e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:24.954320 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-6aa82667-e3c8-4c87-8eb3-f81e4e53fdb6 None None] MFA Rules not processed for user `f6ef1a44226c4ca183d189147cb78413`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:32:24.955918 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-6aa82667-e3c8-4c87-8eb3-f81e4e53fdb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:25.009624 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1257/2554] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:32:24 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:32:25.018967 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceec0cf-e691-4c51-abb2-4d479887b95e None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:25.023044 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-aceec0cf-e691-4c51-abb2-4d479887b95e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:25.049034 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1298/2555] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:32:24 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:32:25.063238 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa82667-e3c8-4c87-8eb3-f81e4e53fdb6 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:25.111300 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceec0cf-e691-4c51-abb2-4d479887b95e None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:25.125932 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dffd80b-8522-4bf6-93f8-0fcda3eff85a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e31f2728056b460fa474478eb85ca1d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:25.125932 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dffd80b-8522-4bf6-93f8-0fcda3eff85a service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:25.125932 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dffd80b-8522-4bf6-93f8-0fcda3eff85a service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:25.125932 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dffd80b-8522-4bf6-93f8-0fcda3eff85a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:25.143079 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dffd80b-8522-4bf6-93f8-0fcda3eff85a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:25.147531 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dffd80b-8522-4bf6-93f8-0fcda3eff85a service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:25.148797 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa82667-e3c8-4c87-8eb3-f81e4e53fdb6 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:25.153894 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeb8248-5665-4638-a642-968678bfad99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:25.155882 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeb8248-5665-4638-a642-968678bfad99 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:25.156003 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeb8248-5665-4638-a642-968678bfad99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:25.156319 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeb8248-5665-4638-a642-968678bfad99 admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:25.157203 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edeb8248-5665-4638-a642-968678bfad99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:25.159221 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edeb8248-5665-4638-a642-968678bfad99 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:25.162619 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1258/2556] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:32:25 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:32:25.255614 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:25.295383 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1299/2557] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:32:25 2022] POST /identity/v3/projects => generated 407 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:32:25.298168 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dffd80b-8522-4bf6-93f8-0fcda3eff85a service cinder] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:25.316989 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dffd80b-8522-4bf6-93f8-0fcda3eff85a service cinder] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:25.396586 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeb8248-5665-4638-a642-968678bfad99 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:25.426499 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b8ab9d-20b1-4608-a0b1-e96731e7d7c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:25.427146 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b8ab9d-20b1-4608-a0b1-e96731e7d7c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:25.427536 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b8ab9d-20b1-4608-a0b1-e96731e7d7c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:25.427884 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b8ab9d-20b1-4608-a0b1-e96731e7d7c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:25.428647 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b8ab9d-20b1-4608-a0b1-e96731e7d7c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:25.430658 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b8ab9d-20b1-4608-a0b1-e96731e7d7c2 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:25.470651 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeb8248-5665-4638-a642-968678bfad99 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:25.475473 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cf97d7-c5a6-44a1-9fb2-de3a74153b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:25.476133 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cf97d7-c5a6-44a1-9fb2-de3a74153b88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:25.476449 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cf97d7-c5a6-44a1-9fb2-de3a74153b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:25.476726 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cf97d7-c5a6-44a1-9fb2-de3a74153b88 admin admin] PATH_INFO: `/v3/users/d5a5475497114d73913ef394d1f8d9a6` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:25.477925 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cf97d7-c5a6-44a1-9fb2-de3a74153b88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5a5475497114d73913ef394d1f8d9a6)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:25.479927 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cf97d7-c5a6-44a1-9fb2-de3a74153b88 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:25.554949 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1259/2558] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:32:25 2022] POST /identity/v3/users => generated 310 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:32:25.558920 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b8ab9d-20b1-4608-a0b1-e96731e7d7c2 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:25.570786 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b8ab9d-20b1-4608-a0b1-e96731e7d7c2 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:25.586113 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b219d56f-9b1a-4503-82fc-8e0f165046ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:25.586754 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b219d56f-9b1a-4503-82fc-8e0f165046ab admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:25.587081 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b219d56f-9b1a-4503-82fc-8e0f165046ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:25.587536 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b219d56f-9b1a-4503-82fc-8e0f165046ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:25.591648 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b219d56f-9b1a-4503-82fc-8e0f165046ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:25.591648 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b219d56f-9b1a-4503-82fc-8e0f165046ab admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:25.604627 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1260/2559] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:32:25 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:32:25.608269 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b219d56f-9b1a-4503-82fc-8e0f165046ab admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:25.619705 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b219d56f-9b1a-4503-82fc-8e0f165046ab admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:25.624690 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c8faa8-3f40-4fda-b856-d5619153fb85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:25.625643 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c8faa8-3f40-4fda-b856-d5619153fb85 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:25.625925 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c8faa8-3f40-4fda-b856-d5619153fb85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:25.626237 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c8faa8-3f40-4fda-b856-d5619153fb85 admin admin] PATH_INFO: `/v3/projects/f289f5bb83a04b619ca44b03ad23bf26/users/154085a5ae3c41e5afb50090e6099851/roles/a432dd4735944ecfab2fa5ce1705d356` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:25.644595 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c8faa8-3f40-4fda-b856-d5619153fb85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f289f5bb83a04b619ca44b03ad23bf26, user_id=154085a5ae3c41e5afb50090e6099851, role_id=a432dd4735944ecfab2fa5ce1705d356)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:25.647746 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c8faa8-3f40-4fda-b856-d5619153fb85 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:25.673186 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-97cf97d7-c5a6-44a1-9fb2-de3a74153b88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5a5475497114d73913ef394d1f8d9a6'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:25.687411 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-97cf97d7-c5a6-44a1-9fb2-de3a74153b88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5a5475497114d73913ef394d1f8d9a6'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:25.688387 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-97cf97d7-c5a6-44a1-9fb2-de3a74153b88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5a5475497114d73913ef394d1f8d9a6'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:25.689052 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-97cf97d7-c5a6-44a1-9fb2-de3a74153b88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5a5475497114d73913ef394d1f8d9a6'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:25.755495 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1261/2560] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:32:25 2022] PUT /identity/v3/projects/f289f5bb83a04b619ca44b03ad23bf26/users/154085a5ae3c41e5afb50090e6099851/roles/a432dd4735944ecfab2fa5ce1705d356 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:32:25.759650 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c8faa8-3f40-4fda-b856-d5619153fb85 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:25.761781 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-97c8faa8-3f40-4fda-b856-d5619153fb85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:25.838748 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-97cf97d7-c5a6-44a1-9fb2-de3a74153b88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5a5475497114d73913ef394d1f8d9a6'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:25.865535 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1300/2561] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:32:25 2022] DELETE /identity/v3/users/d5a5475497114d73913ef394d1f8d9a6 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:32:25.869605 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5171ee6d-4130-4c6e-87df-08dbf0932c36 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:25.869893 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5171ee6d-4130-4c6e-87df-08dbf0932c36 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:25.870200 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5171ee6d-4130-4c6e-87df-08dbf0932c36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:25.884048 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c8faa8-3f40-4fda-b856-d5619153fb85 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:25.897018 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-5171ee6d-4130-4c6e-87df-08dbf0932c36 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:32:25.898270 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-5171ee6d-4130-4c6e-87df-08dbf0932c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:26.011775 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:26.014326 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79574952-a128-4c52-b464-9d747ebe20a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:26.015974 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79574952-a128-4c52-b464-9d747ebe20a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:26.017239 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79574952-a128-4c52-b464-9d747ebe20a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:26.017712 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79574952-a128-4c52-b464-9d747ebe20a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:26.019543 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79574952-a128-4c52-b464-9d747ebe20a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:26.023014 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1301/2562] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:32:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:32:26.023486 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79574952-a128-4c52-b464-9d747ebe20a8 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:26.030204 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5171ee6d-4130-4c6e-87df-08dbf0932c36 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:26.032744 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-5171ee6d-4130-4c6e-87df-08dbf0932c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:26.050406 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1262/2563] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:32:25 2022] GET /identity/v3/roles => generated 1420 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:32:26.050406 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79574952-a128-4c52-b464-9d747ebe20a8 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:26.059181 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79574952-a128-4c52-b464-9d747ebe20a8 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:26.064720 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547d71f8-1781-42f4-9a3e-e6bc6d0e9a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:26.065581 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d71f8-1781-42f4-9a3e-e6bc6d0e9a45 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:26.065941 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d71f8-1781-42f4-9a3e-e6bc6d0e9a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:26.066349 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d71f8-1781-42f4-9a3e-e6bc6d0e9a45 admin admin] PATH_INFO: `/v3/projects/f289f5bb83a04b619ca44b03ad23bf26/users/154085a5ae3c41e5afb50090e6099851/roles/a432dd4735944ecfab2fa5ce1705d356` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:26.082769 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547d71f8-1781-42f4-9a3e-e6bc6d0e9a45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f289f5bb83a04b619ca44b03ad23bf26, user_id=154085a5ae3c41e5afb50090e6099851, role_id=a432dd4735944ecfab2fa5ce1705d356)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:26.092257 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547d71f8-1781-42f4-9a3e-e6bc6d0e9a45 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:26.118383 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1263/2564] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:32:26 2022] PUT /identity/v3/projects/f289f5bb83a04b619ca44b03ad23bf26/users/154085a5ae3c41e5afb50090e6099851/roles/a432dd4735944ecfab2fa5ce1705d356 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:32:26.128458 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ae091-1788-48f6-a6c9-f6d75fd5cee8 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:26.128458 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ae091-1788-48f6-a6c9-f6d75fd5cee8 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:26.128458 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ae091-1788-48f6-a6c9-f6d75fd5cee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:26.134740 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5171ee6d-4130-4c6e-87df-08dbf0932c36 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:26.161503 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-d90ae091-1788-48f6-a6c9-f6d75fd5cee8 None None] MFA Rules not processed for user `93956198082e44189d090d4c5fb95b63`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:32:26.163194 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-d90ae091-1788-48f6-a6c9-f6d75fd5cee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:26.190966 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9057ec-6b3b-4977-ae1b-7ad552e80f4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:26.191983 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9057ec-6b3b-4977-ae1b-7ad552e80f4f service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:26.192315 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9057ec-6b3b-4977-ae1b-7ad552e80f4f service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:26.192595 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9057ec-6b3b-4977-ae1b-7ad552e80f4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:26.249508 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1264/2565] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:32:26 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:32:26.251732 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eddcd1-f732-42cb-81fb-f4851090309f None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:26.252125 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eddcd1-f732-42cb-81fb-f4851090309f None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:26.252515 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eddcd1-f732-42cb-81fb-f4851090309f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:26.277615 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9057ec-6b3b-4977-ae1b-7ad552e80f4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:26.279101 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9057ec-6b3b-4977-ae1b-7ad552e80f4f service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:26.289557 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-d1eddcd1-f732-42cb-81fb-f4851090309f None None] MFA Rules not processed for user `154085a5ae3c41e5afb50090e6099851`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:32:26.291166 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-d1eddcd1-f732-42cb-81fb-f4851090309f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:26.292970 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1302/2566] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:32:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:32:26.293865 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9057ec-6b3b-4977-ae1b-7ad552e80f4f service neutron] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:26.295228 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-5d9057ec-6b3b-4977-ae1b-7ad552e80f4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:26.395601 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9057ec-6b3b-4977-ae1b-7ad552e80f4f service neutron] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:26.402324 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca1260e-7cac-499a-ad91-6013e180c00a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a2ce1df87fa47e5be301916bf3245fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:26.403067 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca1260e-7cac-499a-ad91-6013e180c00a service nova] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:26.403450 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca1260e-7cac-499a-ad91-6013e180c00a service nova] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:26.403799 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca1260e-7cac-499a-ad91-6013e180c00a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:26.404603 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1265/2567] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:32:26 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:32:26.409816 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-aca1260e-7cac-499a-ad91-6013e180c00a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:26.420006 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eddcd1-f732-42cb-81fb-f4851090309f None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:26.423336 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-d1eddcd1-f732-42cb-81fb-f4851090309f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:26.491091 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca1260e-7cac-499a-ad91-6013e180c00a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:26.493899 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca1260e-7cac-499a-ad91-6013e180c00a service nova] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:26.494898 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eddcd1-f732-42cb-81fb-f4851090309f None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:26.498798 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b640549-dc21-425a-9a93-82df4aaa6d49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e31f2728056b460fa474478eb85ca1d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:26.499588 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b640549-dc21-425a-9a93-82df4aaa6d49 service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:26.500770 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b640549-dc21-425a-9a93-82df4aaa6d49 service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:26.501060 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b640549-dc21-425a-9a93-82df4aaa6d49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:26.505406 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1303/2568] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:32:26 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:32:26.519684 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b640549-dc21-425a-9a93-82df4aaa6d49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:26.522137 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b640549-dc21-425a-9a93-82df4aaa6d49 service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:26.548713 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1266/2569] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:32:26 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:32:26.865646 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca1260e-7cac-499a-ad91-6013e180c00a service nova] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:26.867467 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-aca1260e-7cac-499a-ad91-6013e180c00a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:26.881765 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca1260e-7cac-499a-ad91-6013e180c00a service nova] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:26.887929 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be519ad3-fd9e-4403-98e1-0f8f1e9ccedc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:26.891404 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be519ad3-fd9e-4403-98e1-0f8f1e9ccedc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:26.891404 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be519ad3-fd9e-4403-98e1-0f8f1e9ccedc admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:26.891404 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be519ad3-fd9e-4403-98e1-0f8f1e9ccedc admin admin] PATH_INFO: `/v3/projects/750fe45bcd0649759bdbcde0c683acab` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:26.905659 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be519ad3-fd9e-4403-98e1-0f8f1e9ccedc admin admin] RBAC: Authorizing `identity:delete_project(project_id=750fe45bcd0649759bdbcde0c683acab)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:26.910565 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be519ad3-fd9e-4403-98e1-0f8f1e9ccedc admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:26.982495 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b640549-dc21-425a-9a93-82df4aaa6d49 service cinder] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:26.984022 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-1b640549-dc21-425a-9a93-82df4aaa6d49 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:26.996753 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b640549-dc21-425a-9a93-82df4aaa6d49 service cinder] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:27.001421 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7127e4ce-973d-4588-af72-d34ee9571054 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e31f2728056b460fa474478eb85ca1d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:27.002050 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7127e4ce-973d-4588-af72-d34ee9571054 service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:27.002339 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7127e4ce-973d-4588-af72-d34ee9571054 service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:27.002635 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7127e4ce-973d-4588-af72-d34ee9571054 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:27.004587 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-7127e4ce-973d-4588-af72-d34ee9571054 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:27.074584 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-be519ad3-fd9e-4403-98e1-0f8f1e9ccedc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '750fe45bcd0649759bdbcde0c683acab', 'request_id': 'req-be519ad3-fd9e-4403-98e1-0f8f1e9ccedc'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:27.075688 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7127e4ce-973d-4588-af72-d34ee9571054 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:27.076470 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-be519ad3-fd9e-4403-98e1-0f8f1e9ccedc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '750fe45bcd0649759bdbcde0c683acab', 'request_id': 'req-be519ad3-fd9e-4403-98e1-0f8f1e9ccedc'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:27.077233 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7127e4ce-973d-4588-af72-d34ee9571054 service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:27.087691 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1267/2570] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:32:26 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:32:27.130477 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:27.196174 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-be519ad3-fd9e-4403-98e1-0f8f1e9ccedc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '750fe45bcd0649759bdbcde0c683acab', 'request_id': 'req-be519ad3-fd9e-4403-98e1-0f8f1e9ccedc'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:27.209196 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-be519ad3-fd9e-4403-98e1-0f8f1e9ccedc admin admin] The token cache is being invalidate because project 750fe45bcd0649759bdbcde0c683acab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:32:27.209489 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-be519ad3-fd9e-4403-98e1-0f8f1e9ccedc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:27.210208 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-be519ad3-fd9e-4403-98e1-0f8f1e9ccedc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:27.214661 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1304/2571] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:32:26 2022] DELETE /identity/v3/projects/750fe45bcd0649759bdbcde0c683acab => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:32:27.215842 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7127e4ce-973d-4588-af72-d34ee9571054 service cinder] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:27.228499 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7127e4ce-973d-4588-af72-d34ee9571054 service cinder] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:27.328698 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850e8d46-b8ec-4866-84c9-79d7956c80d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:27.329607 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850e8d46-b8ec-4866-84c9-79d7956c80d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:27.329936 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850e8d46-b8ec-4866-84c9-79d7956c80d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:27.330216 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850e8d46-b8ec-4866-84c9-79d7956c80d0 admin admin] PATH_INFO: `/v3/users/91dfaa1fdb224a059b6e4bb1aa4348e2` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:27.331488 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850e8d46-b8ec-4866-84c9-79d7956c80d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91dfaa1fdb224a059b6e4bb1aa4348e2)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:27.333899 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850e8d46-b8ec-4866-84c9-79d7956c80d0 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:27.500714 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-850e8d46-b8ec-4866-84c9-79d7956c80d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91dfaa1fdb224a059b6e4bb1aa4348e2'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:27.513057 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-850e8d46-b8ec-4866-84c9-79d7956c80d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91dfaa1fdb224a059b6e4bb1aa4348e2'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:27.513766 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-850e8d46-b8ec-4866-84c9-79d7956c80d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91dfaa1fdb224a059b6e4bb1aa4348e2'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:27.521049 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-850e8d46-b8ec-4866-84c9-79d7956c80d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91dfaa1fdb224a059b6e4bb1aa4348e2'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:27.521652 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-850e8d46-b8ec-4866-84c9-79d7956c80d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91dfaa1fdb224a059b6e4bb1aa4348e2'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:27.610407 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1268/2572] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:32:27 2022] DELETE /identity/v3/users/91dfaa1fdb224a059b6e4bb1aa4348e2 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:32:27.893344 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:28.184401 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be519ad3-fd9e-4403-98e1-0f8f1e9ccedc admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:28.186011 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-be519ad3-fd9e-4403-98e1-0f8f1e9ccedc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:28.200767 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be519ad3-fd9e-4403-98e1-0f8f1e9ccedc admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:28.271108 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dced5b15-f5f2-4729-b8b1-5542c03d8b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:28.272123 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dced5b15-f5f2-4729-b8b1-5542c03d8b83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:28.272570 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dced5b15-f5f2-4729-b8b1-5542c03d8b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:28.273022 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dced5b15-f5f2-4729-b8b1-5542c03d8b83 admin admin] PATH_INFO: `/v3/projects/5d8df6a1be084e44909e1e482298abaf` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:28.274642 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dced5b15-f5f2-4729-b8b1-5542c03d8b83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d8df6a1be084e44909e1e482298abaf)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:28.279661 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dced5b15-f5f2-4729-b8b1-5542c03d8b83 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:28.416443 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-dced5b15-f5f2-4729-b8b1-5542c03d8b83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d8df6a1be084e44909e1e482298abaf', 'request_id': 'req-dced5b15-f5f2-4729-b8b1-5542c03d8b83'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:28.427490 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-dced5b15-f5f2-4729-b8b1-5542c03d8b83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d8df6a1be084e44909e1e482298abaf', 'request_id': 'req-dced5b15-f5f2-4729-b8b1-5542c03d8b83'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:28.510823 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-dced5b15-f5f2-4729-b8b1-5542c03d8b83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d8df6a1be084e44909e1e482298abaf', 'request_id': 'req-dced5b15-f5f2-4729-b8b1-5542c03d8b83'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:28.525082 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-dced5b15-f5f2-4729-b8b1-5542c03d8b83 admin admin] The token cache is being invalidate because project 5d8df6a1be084e44909e1e482298abaf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:32:28.525368 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-dced5b15-f5f2-4729-b8b1-5542c03d8b83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:28.526388 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-dced5b15-f5f2-4729-b8b1-5542c03d8b83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:28.528256 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1305/2573] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:32:28 2022] DELETE /identity/v3/projects/5d8df6a1be084e44909e1e482298abaf => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:32:28.533735 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df977ba8-7465-46d4-b784-fe02e77e3e36 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:28.534229 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df977ba8-7465-46d4-b784-fe02e77e3e36 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:28.534229 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df977ba8-7465-46d4-b784-fe02e77e3e36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:28.556416 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-df977ba8-7465-46d4-b784-fe02e77e3e36 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:32:28.557637 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-df977ba8-7465-46d4-b784-fe02e77e3e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:28.649295 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1269/2574] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:32:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:32:28.666037 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eae21e-0d34-4d2b-a18c-121e01780979 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:28.666331 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eae21e-0d34-4d2b-a18c-121e01780979 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:28.666661 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eae21e-0d34-4d2b-a18c-121e01780979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:28.699055 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-44eae21e-0d34-4d2b-a18c-121e01780979 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:32:28.700402 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-44eae21e-0d34-4d2b-a18c-121e01780979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:28.708585 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1306/2575] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:32:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:32:28.713852 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df977ba8-7465-46d4-b784-fe02e77e3e36 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:28.735182 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df977ba8-7465-46d4-b784-fe02e77e3e36 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:28.741816 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d412c33-4778-4256-89b4-0aac96d43588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:28.742470 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d412c33-4778-4256-89b4-0aac96d43588 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:28.742756 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d412c33-4778-4256-89b4-0aac96d43588 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:28.743032 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d412c33-4778-4256-89b4-0aac96d43588 admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:28.743950 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d412c33-4778-4256-89b4-0aac96d43588 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:32:28.744056 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d412c33-4778-4256-89b4-0aac96d43588 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:28.745693 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d412c33-4778-4256-89b4-0aac96d43588 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:28.762350 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1270/2576] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Aug 4 21:32:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:32:28.766908 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eae21e-0d34-4d2b-a18c-121e01780979 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:28.777206 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eae21e-0d34-4d2b-a18c-121e01780979 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:28.783359 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca2d81d-8dc4-4c25-8009-26e3324db2a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:28.791521 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca2d81d-8dc4-4c25-8009-26e3324db2a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:28.791521 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca2d81d-8dc4-4c25-8009-26e3324db2a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:28.791521 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca2d81d-8dc4-4c25-8009-26e3324db2a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:28.791521 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca2d81d-8dc4-4c25-8009-26e3324db2a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:28.791521 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca2d81d-8dc4-4c25-8009-26e3324db2a2 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:28.908625 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1307/2577] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:32:28 2022] POST /identity/v3/projects => generated 361 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:32:28.912672 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d412c33-4778-4256-89b4-0aac96d43588 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:28.927593 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d412c33-4778-4256-89b4-0aac96d43588 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:29.004623 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922b4f74-53cd-4bcc-8164-78634791a9ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:29.005762 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922b4f74-53cd-4bcc-8164-78634791a9ed admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:29.006041 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922b4f74-53cd-4bcc-8164-78634791a9ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:29.006541 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922b4f74-53cd-4bcc-8164-78634791a9ed admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:29.007223 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922b4f74-53cd-4bcc-8164-78634791a9ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:29.009406 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922b4f74-53cd-4bcc-8164-78634791a9ed admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:29.174104 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1271/2578] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:32:28 2022] POST /identity/v3/users => generated 281 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:32:29.179076 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca2d81d-8dc4-4c25-8009-26e3324db2a2 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:29.190610 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca2d81d-8dc4-4c25-8009-26e3324db2a2 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:29.199194 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:29.200246 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8341588-2db2-4a39-b882-18f4ed9d16a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:29.201308 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8341588-2db2-4a39-b882-18f4ed9d16a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:29.201892 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8341588-2db2-4a39-b882-18f4ed9d16a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:29.202374 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8341588-2db2-4a39-b882-18f4ed9d16a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:29.203808 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8341588-2db2-4a39-b882-18f4ed9d16a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:29.206446 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8341588-2db2-4a39-b882-18f4ed9d16a9 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:29.220430 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1308/2579] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:32:29 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:32:29.226113 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922b4f74-53cd-4bcc-8164-78634791a9ed admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:29.246883 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922b4f74-53cd-4bcc-8164-78634791a9ed admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:29.253767 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1ac77d-5612-493a-8a06-2f344bb68e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:29.254717 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1ac77d-5612-493a-8a06-2f344bb68e0c admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:29.255006 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1ac77d-5612-493a-8a06-2f344bb68e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:29.255274 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1ac77d-5612-493a-8a06-2f344bb68e0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:29.256093 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1ac77d-5612-493a-8a06-2f344bb68e0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:29.257838 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1ac77d-5612-493a-8a06-2f344bb68e0c admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:29.269782 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1272/2580] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:32:29 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:32:29.274150 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8341588-2db2-4a39-b882-18f4ed9d16a9 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:29.282638 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:29.286571 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8341588-2db2-4a39-b882-18f4ed9d16a9 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:29.292749 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa5a532-7776-48b6-a057-3cb5578f78eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:29.293653 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa5a532-7776-48b6-a057-3cb5578f78eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:29.294007 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa5a532-7776-48b6-a057-3cb5578f78eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:29.294329 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa5a532-7776-48b6-a057-3cb5578f78eb admin admin] PATH_INFO: `/v3/projects/a31e71e1448d418bb308b543c99d7c35/users/d3d955ceeb694046af4ecfdbb9d673bc/roles/8feebc60363c4feb870718b1721e5324` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:29.324597 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa5a532-7776-48b6-a057-3cb5578f78eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=a31e71e1448d418bb308b543c99d7c35, user_id=d3d955ceeb694046af4ecfdbb9d673bc, role_id=8feebc60363c4feb870718b1721e5324)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:29.327743 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa5a532-7776-48b6-a057-3cb5578f78eb admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:29.439182 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1309/2581] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:32:29 2022] PUT /identity/v3/projects/a31e71e1448d418bb308b543c99d7c35/users/d3d955ceeb694046af4ecfdbb9d673bc/roles/8feebc60363c4feb870718b1721e5324 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:32:29.450359 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dceb151-91e7-47a8-9f1b-75ac45ba20b4 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:29.450647 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dceb151-91e7-47a8-9f1b-75ac45ba20b4 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:29.450925 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dceb151-91e7-47a8-9f1b-75ac45ba20b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:29.482049 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-4dceb151-91e7-47a8-9f1b-75ac45ba20b4 None None] MFA Rules not processed for user `d3d955ceeb694046af4ecfdbb9d673bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:32:29.483511 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-4dceb151-91e7-47a8-9f1b-75ac45ba20b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:29.564385 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1273/2582] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:32:29 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:32:29.569285 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa5a532-7776-48b6-a057-3cb5578f78eb admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:29.650883 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa5a532-7776-48b6-a057-3cb5578f78eb admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:29.655934 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13941af-921a-4308-aba6-470d04fd9d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:29.656614 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13941af-921a-4308-aba6-470d04fd9d37 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:29.656901 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13941af-921a-4308-aba6-470d04fd9d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:29.657182 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13941af-921a-4308-aba6-470d04fd9d37 admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:29.657858 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13941af-921a-4308-aba6-470d04fd9d37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:29.672638 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13941af-921a-4308-aba6-470d04fd9d37 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:29.759639 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1310/2583] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:32:29 2022] POST /identity/v3/projects => generated 363 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:32:29.761149 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dceb151-91e7-47a8-9f1b-75ac45ba20b4 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:29.776890 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dceb151-91e7-47a8-9f1b-75ac45ba20b4 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:29.842914 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5372247f-7bed-4726-a8f4-76ebd8c6a3cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:29.843618 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5372247f-7bed-4726-a8f4-76ebd8c6a3cf admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:29.843910 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5372247f-7bed-4726-a8f4-76ebd8c6a3cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:29.844541 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5372247f-7bed-4726-a8f4-76ebd8c6a3cf admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:29.845250 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5372247f-7bed-4726-a8f4-76ebd8c6a3cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:29.847341 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5372247f-7bed-4726-a8f4-76ebd8c6a3cf admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:29.943682 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1274/2584] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:32:29 2022] POST /identity/v3/users => generated 288 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:32:29.947548 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13941af-921a-4308-aba6-470d04fd9d37 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:29.978172 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13941af-921a-4308-aba6-470d04fd9d37 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:30.002137 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4ed6f7-bf41-41f1-858a-de8e185d9784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:30.002859 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4ed6f7-bf41-41f1-858a-de8e185d9784 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:30.007836 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4ed6f7-bf41-41f1-858a-de8e185d9784 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:30.007836 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4ed6f7-bf41-41f1-858a-de8e185d9784 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:30.007836 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4ed6f7-bf41-41f1-858a-de8e185d9784 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:30.007836 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4ed6f7-bf41-41f1-858a-de8e185d9784 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:30.023352 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1311/2585] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:32:29 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:32:30.027785 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5372247f-7bed-4726-a8f4-76ebd8c6a3cf admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:30.041063 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5372247f-7bed-4726-a8f4-76ebd8c6a3cf admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:30.046285 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e5c279-ffd6-47ae-a609-39e831d8fb39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:30.047623 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e5c279-ffd6-47ae-a609-39e831d8fb39 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:30.047965 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e5c279-ffd6-47ae-a609-39e831d8fb39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:30.048434 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e5c279-ffd6-47ae-a609-39e831d8fb39 admin admin] PATH_INFO: `/v3/projects/e0bd6f387595440fa1b35dfb6d0b878a/users/ec09eea2aab5465c9b0572aa9c9b815b/roles/a432dd4735944ecfab2fa5ce1705d356` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:30.070213 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e5c279-ffd6-47ae-a609-39e831d8fb39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0bd6f387595440fa1b35dfb6d0b878a, user_id=ec09eea2aab5465c9b0572aa9c9b815b, role_id=a432dd4735944ecfab2fa5ce1705d356)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:30.073452 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e5c279-ffd6-47ae-a609-39e831d8fb39 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:30.157304 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1275/2586] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:32:30 2022] PUT /identity/v3/projects/e0bd6f387595440fa1b35dfb6d0b878a/users/ec09eea2aab5465c9b0572aa9c9b815b/roles/a432dd4735944ecfab2fa5ce1705d356 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:32:30.161660 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4ed6f7-bf41-41f1-858a-de8e185d9784 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:30.172967 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4ed6f7-bf41-41f1-858a-de8e185d9784 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:30.245823 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a877e6-4d04-4434-834a-4d9e499de733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:30.248897 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a877e6-4d04-4434-834a-4d9e499de733 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:30.248897 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a877e6-4d04-4434-834a-4d9e499de733 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:30.248897 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a877e6-4d04-4434-834a-4d9e499de733 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:30.248897 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a877e6-4d04-4434-834a-4d9e499de733 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:30.250526 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a877e6-4d04-4434-834a-4d9e499de733 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:30.261380 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1312/2587] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:32:30 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:32:30.265795 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e5c279-ffd6-47ae-a609-39e831d8fb39 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:30.277804 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e5c279-ffd6-47ae-a609-39e831d8fb39 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:30.283362 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da001aa-f4dc-44df-b5d4-9b84faf866e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:30.284057 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da001aa-f4dc-44df-b5d4-9b84faf866e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:30.284346 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da001aa-f4dc-44df-b5d4-9b84faf866e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:30.284639 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da001aa-f4dc-44df-b5d4-9b84faf866e6 admin admin] PATH_INFO: `/v3/projects/e0bd6f387595440fa1b35dfb6d0b878a/users/ec09eea2aab5465c9b0572aa9c9b815b/roles/a432dd4735944ecfab2fa5ce1705d356` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:30.286711 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da001aa-f4dc-44df-b5d4-9b84faf866e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0bd6f387595440fa1b35dfb6d0b878a, user_id=ec09eea2aab5465c9b0572aa9c9b815b, role_id=a432dd4735944ecfab2fa5ce1705d356)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:30.289476 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da001aa-f4dc-44df-b5d4-9b84faf866e6 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:30.301015 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1276/2588] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:32:30 2022] PUT /identity/v3/projects/e0bd6f387595440fa1b35dfb6d0b878a/users/ec09eea2aab5465c9b0572aa9c9b815b/roles/a432dd4735944ecfab2fa5ce1705d356 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:32:30.312205 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e7e647-30bb-4d71-b416-196063ac2b39 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:30.312529 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e7e647-30bb-4d71-b416-196063ac2b39 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:30.312807 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e7e647-30bb-4d71-b416-196063ac2b39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:30.342282 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-69e7e647-30bb-4d71-b416-196063ac2b39 None None] MFA Rules not processed for user `ec09eea2aab5465c9b0572aa9c9b815b`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:32:30.344987 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-69e7e647-30bb-4d71-b416-196063ac2b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:30.416378 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1313/2589] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:32:30 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:32:30.655045 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da001aa-f4dc-44df-b5d4-9b84faf866e6 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:30.657332 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-6da001aa-f4dc-44df-b5d4-9b84faf866e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:30.689426 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:30.734426 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da001aa-f4dc-44df-b5d4-9b84faf866e6 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:30.740797 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c0f486-39c0-4b12-9837-afe51804fb7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e31f2728056b460fa474478eb85ca1d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:30.741914 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c0f486-39c0-4b12-9837-afe51804fb7f service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:30.742405 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c0f486-39c0-4b12-9837-afe51804fb7f service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:30.743114 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c0f486-39c0-4b12-9837-afe51804fb7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:30.757672 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c0f486-39c0-4b12-9837-afe51804fb7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:30.759131 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c0f486-39c0-4b12-9837-afe51804fb7f service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:30.770164 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1277/2590] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:32:30 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:32:31.441412 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:31.592271 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:31.717483 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e7e647-30bb-4d71-b416-196063ac2b39 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:31.787007 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e7e647-30bb-4d71-b416-196063ac2b39 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:31.791947 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cbb2dc-e120-44ad-b5ad-53a6b5885023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:31.805128 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cbb2dc-e120-44ad-b5ad-53a6b5885023 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:31.805128 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cbb2dc-e120-44ad-b5ad-53a6b5885023 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:31.805128 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cbb2dc-e120-44ad-b5ad-53a6b5885023 admin admin] PATH_INFO: `/v3/users/d3d955ceeb694046af4ecfdbb9d673bc` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:31.805128 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cbb2dc-e120-44ad-b5ad-53a6b5885023 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3d955ceeb694046af4ecfdbb9d673bc)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:31.805128 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cbb2dc-e120-44ad-b5ad-53a6b5885023 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:31.970769 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-69cbb2dc-e120-44ad-b5ad-53a6b5885023 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3d955ceeb694046af4ecfdbb9d673bc'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:31.982775 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-69cbb2dc-e120-44ad-b5ad-53a6b5885023 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3d955ceeb694046af4ecfdbb9d673bc'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:31.983501 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-69cbb2dc-e120-44ad-b5ad-53a6b5885023 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3d955ceeb694046af4ecfdbb9d673bc'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:31.984118 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-69cbb2dc-e120-44ad-b5ad-53a6b5885023 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3d955ceeb694046af4ecfdbb9d673bc'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:32.080981 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-69cbb2dc-e120-44ad-b5ad-53a6b5885023 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3d955ceeb694046af4ecfdbb9d673bc'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:32.092978 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1314/2591] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:32:31 2022] DELETE /identity/v3/users/d3d955ceeb694046af4ecfdbb9d673bc => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:32:32.095896 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760d4035-0148-4d1f-8489-0abb72fb60c0 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:32.096359 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760d4035-0148-4d1f-8489-0abb72fb60c0 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:32.096497 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760d4035-0148-4d1f-8489-0abb72fb60c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:32.117109 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-760d4035-0148-4d1f-8489-0abb72fb60c0 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:32:32.118333 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-760d4035-0148-4d1f-8489-0abb72fb60c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:32.181172 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1278/2592] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:32:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:32:32.191672 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cbb2dc-e120-44ad-b5ad-53a6b5885023 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:32.193131 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-69cbb2dc-e120-44ad-b5ad-53a6b5885023 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:32.252457 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cbb2dc-e120-44ad-b5ad-53a6b5885023 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:32.256806 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb33493-5154-43dd-b0b5-2cbdbca30f55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:32.257453 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb33493-5154-43dd-b0b5-2cbdbca30f55 service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:32.257729 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb33493-5154-43dd-b0b5-2cbdbca30f55 service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:32.258237 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb33493-5154-43dd-b0b5-2cbdbca30f55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:32.267540 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:32.268282 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb33493-5154-43dd-b0b5-2cbdbca30f55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:32.269493 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb33493-5154-43dd-b0b5-2cbdbca30f55 service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:32.279391 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1315/2593] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:32:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:32:32.364485 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:32.674173 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760d4035-0148-4d1f-8489-0abb72fb60c0 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:32.674368 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:32.676129 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-760d4035-0148-4d1f-8489-0abb72fb60c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:32.689002 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760d4035-0148-4d1f-8489-0abb72fb60c0 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:32.693821 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358d6073-c83e-4390-a9cc-97004676ac82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:32.694454 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358d6073-c83e-4390-a9cc-97004676ac82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:32.694739 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358d6073-c83e-4390-a9cc-97004676ac82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:32.695044 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358d6073-c83e-4390-a9cc-97004676ac82 admin admin] PATH_INFO: `/v3/projects/a31e71e1448d418bb308b543c99d7c35` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:32.696479 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358d6073-c83e-4390-a9cc-97004676ac82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a31e71e1448d418bb308b543c99d7c35)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:32.698399 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358d6073-c83e-4390-a9cc-97004676ac82 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:32.822850 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-358d6073-c83e-4390-a9cc-97004676ac82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a31e71e1448d418bb308b543c99d7c35', 'request_id': 'req-358d6073-c83e-4390-a9cc-97004676ac82'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:32.835239 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-358d6073-c83e-4390-a9cc-97004676ac82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a31e71e1448d418bb308b543c99d7c35', 'request_id': 'req-358d6073-c83e-4390-a9cc-97004676ac82'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:32.912663 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-358d6073-c83e-4390-a9cc-97004676ac82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a31e71e1448d418bb308b543c99d7c35', 'request_id': 'req-358d6073-c83e-4390-a9cc-97004676ac82'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:32.925556 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-358d6073-c83e-4390-a9cc-97004676ac82 admin admin] The token cache is being invalidate because project a31e71e1448d418bb308b543c99d7c35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:32:32.925845 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-358d6073-c83e-4390-a9cc-97004676ac82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:32.926514 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-358d6073-c83e-4390-a9cc-97004676ac82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:32.928075 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1279/2594] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:32:32 2022] DELETE /identity/v3/projects/a31e71e1448d418bb308b543c99d7c35 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:32:32.932088 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb33493-5154-43dd-b0b5-2cbdbca30f55 service neutron] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:32.942591 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb33493-5154-43dd-b0b5-2cbdbca30f55 service neutron] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:33.002429 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aef8f0f-0443-4059-aa2b-85cd3563b1bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:33.003169 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aef8f0f-0443-4059-aa2b-85cd3563b1bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:33.003771 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aef8f0f-0443-4059-aa2b-85cd3563b1bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:33.004059 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aef8f0f-0443-4059-aa2b-85cd3563b1bc admin admin] PATH_INFO: `/v3/users/ec09eea2aab5465c9b0572aa9c9b815b` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:33.005531 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aef8f0f-0443-4059-aa2b-85cd3563b1bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec09eea2aab5465c9b0572aa9c9b815b)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:33.007584 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aef8f0f-0443-4059-aa2b-85cd3563b1bc admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:33.020045 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:33.252081 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-3aef8f0f-0443-4059-aa2b-85cd3563b1bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec09eea2aab5465c9b0572aa9c9b815b'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:33.266313 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-3aef8f0f-0443-4059-aa2b-85cd3563b1bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec09eea2aab5465c9b0572aa9c9b815b'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:33.266849 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-3aef8f0f-0443-4059-aa2b-85cd3563b1bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec09eea2aab5465c9b0572aa9c9b815b'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:33.268193 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-3aef8f0f-0443-4059-aa2b-85cd3563b1bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec09eea2aab5465c9b0572aa9c9b815b'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:33.332918 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-3aef8f0f-0443-4059-aa2b-85cd3563b1bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec09eea2aab5465c9b0572aa9c9b815b'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:33.346556 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1316/2595] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:32:32 2022] DELETE /identity/v3/users/ec09eea2aab5465c9b0572aa9c9b815b => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:32:33.636134 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:33.681613 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:33.843604 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358d6073-c83e-4390-a9cc-97004676ac82 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:33.845033 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-358d6073-c83e-4390-a9cc-97004676ac82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:33.860341 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358d6073-c83e-4390-a9cc-97004676ac82 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:33.924881 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1f4fff-fa7c-4360-8fbc-f3cffb50ab78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:33.925542 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1f4fff-fa7c-4360-8fbc-f3cffb50ab78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:33.925821 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1f4fff-fa7c-4360-8fbc-f3cffb50ab78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:33.926096 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1f4fff-fa7c-4360-8fbc-f3cffb50ab78 admin admin] PATH_INFO: `/v3/projects/e0bd6f387595440fa1b35dfb6d0b878a` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:33.927266 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1f4fff-fa7c-4360-8fbc-f3cffb50ab78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0bd6f387595440fa1b35dfb6d0b878a)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:33.929232 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1f4fff-fa7c-4360-8fbc-f3cffb50ab78 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:34.054419 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-7a1f4fff-fa7c-4360-8fbc-f3cffb50ab78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0bd6f387595440fa1b35dfb6d0b878a', 'request_id': 'req-7a1f4fff-fa7c-4360-8fbc-f3cffb50ab78'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:34.060274 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-7a1f4fff-fa7c-4360-8fbc-f3cffb50ab78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0bd6f387595440fa1b35dfb6d0b878a', 'request_id': 'req-7a1f4fff-fa7c-4360-8fbc-f3cffb50ab78'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:34.062986 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:34.144105 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-7a1f4fff-fa7c-4360-8fbc-f3cffb50ab78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0bd6f387595440fa1b35dfb6d0b878a', 'request_id': 'req-7a1f4fff-fa7c-4360-8fbc-f3cffb50ab78'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:34.155517 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-7a1f4fff-fa7c-4360-8fbc-f3cffb50ab78 admin admin] The token cache is being invalidate because project e0bd6f387595440fa1b35dfb6d0b878a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:32:34.155835 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-7a1f4fff-fa7c-4360-8fbc-f3cffb50ab78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:34.156568 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-7a1f4fff-fa7c-4360-8fbc-f3cffb50ab78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:34.158764 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1280/2596] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:32:33 2022] DELETE /identity/v3/projects/e0bd6f387595440fa1b35dfb6d0b878a => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:32:34.163890 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e3e2b7-238e-4848-b2c8-1c973ebd551e None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:34.164184 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e3e2b7-238e-4848-b2c8-1c973ebd551e None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:34.164464 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e3e2b7-238e-4848-b2c8-1c973ebd551e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:34.184254 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-f0e3e2b7-238e-4848-b2c8-1c973ebd551e None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:32:34.185460 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-f0e3e2b7-238e-4848-b2c8-1c973ebd551e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:34.249735 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:34.253926 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1317/2597] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:32:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:32:34.262406 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14bb103-b6c8-471c-91ee-6581ff9cb33f None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:34.262699 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14bb103-b6c8-471c-91ee-6581ff9cb33f None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:34.263015 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14bb103-b6c8-471c-91ee-6581ff9cb33f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:34.283516 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-c14bb103-b6c8-471c-91ee-6581ff9cb33f None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:32:34.287076 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-c14bb103-b6c8-471c-91ee-6581ff9cb33f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:34.293025 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1281/2598] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:32:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:32:34.297548 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e3e2b7-238e-4848-b2c8-1c973ebd551e None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:34.308577 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e3e2b7-238e-4848-b2c8-1c973ebd551e None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:34.313373 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae6a3e9-57eb-48ad-b31a-2cf0f9f3564c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:34.313949 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae6a3e9-57eb-48ad-b31a-2cf0f9f3564c admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:34.314229 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae6a3e9-57eb-48ad-b31a-2cf0f9f3564c admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:34.314502 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae6a3e9-57eb-48ad-b31a-2cf0f9f3564c admin admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:34.315837 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae6a3e9-57eb-48ad-b31a-2cf0f9f3564c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:32:34.316141 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae6a3e9-57eb-48ad-b31a-2cf0f9f3564c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:34.317760 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae6a3e9-57eb-48ad-b31a-2cf0f9f3564c admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:34.329841 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1318/2599] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Aug 4 21:32:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:32:34.334662 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14bb103-b6c8-471c-91ee-6581ff9cb33f None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:34.340911 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:34.344036 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14bb103-b6c8-471c-91ee-6581ff9cb33f None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:34.348822 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578b7587-2e23-4f07-85e6-ee0213ba62dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:34.349414 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578b7587-2e23-4f07-85e6-ee0213ba62dd admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:34.349698 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578b7587-2e23-4f07-85e6-ee0213ba62dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:34.353203 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578b7587-2e23-4f07-85e6-ee0213ba62dd admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:34.353203 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578b7587-2e23-4f07-85e6-ee0213ba62dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:34.353203 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578b7587-2e23-4f07-85e6-ee0213ba62dd admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:34.470179 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1282/2600] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:32:34 2022] POST /identity/v3/projects => generated 375 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:32:34.475400 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae6a3e9-57eb-48ad-b31a-2cf0f9f3564c admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:34.487357 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae6a3e9-57eb-48ad-b31a-2cf0f9f3564c admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:34.560848 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b9a451-5d79-4946-a1e2-b88a6989e443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:34.561490 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b9a451-5d79-4946-a1e2-b88a6989e443 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:34.561769 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b9a451-5d79-4946-a1e2-b88a6989e443 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:34.562044 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b9a451-5d79-4946-a1e2-b88a6989e443 admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:34.562718 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b9a451-5d79-4946-a1e2-b88a6989e443 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:34.564805 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b9a451-5d79-4946-a1e2-b88a6989e443 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:34.677226 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1319/2601] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:32:34 2022] POST /identity/v3/users => generated 288 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:32:34.681465 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578b7587-2e23-4f07-85e6-ee0213ba62dd admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:34.691860 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578b7587-2e23-4f07-85e6-ee0213ba62dd admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:34.696631 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2811e34-4734-4551-a6d1-9856ccb42e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:34.697238 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2811e34-4734-4551-a6d1-9856ccb42e43 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:34.697517 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2811e34-4734-4551-a6d1-9856ccb42e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:34.697791 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2811e34-4734-4551-a6d1-9856ccb42e43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:34.698485 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2811e34-4734-4551-a6d1-9856ccb42e43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:34.700144 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2811e34-4734-4551-a6d1-9856ccb42e43 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:34.710370 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1283/2602] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:32:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:32:34.717909 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b9a451-5d79-4946-a1e2-b88a6989e443 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:34.733365 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b9a451-5d79-4946-a1e2-b88a6989e443 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:34.740583 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34a4e58-1df1-42b7-b1ed-ff2481a86892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:34.740583 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34a4e58-1df1-42b7-b1ed-ff2481a86892 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:34.740583 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34a4e58-1df1-42b7-b1ed-ff2481a86892 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:34.740583 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34a4e58-1df1-42b7-b1ed-ff2481a86892 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:34.746044 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34a4e58-1df1-42b7-b1ed-ff2481a86892 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:34.746044 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34a4e58-1df1-42b7-b1ed-ff2481a86892 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:34.763990 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1320/2603] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:32:34 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:32:34.769082 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2811e34-4734-4551-a6d1-9856ccb42e43 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:34.791103 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2811e34-4734-4551-a6d1-9856ccb42e43 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:34.796218 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e83c28c-b7a9-4b8e-ad24-32c9139f8c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:34.796901 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e83c28c-b7a9-4b8e-ad24-32c9139f8c55 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:34.797184 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e83c28c-b7a9-4b8e-ad24-32c9139f8c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:34.797453 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e83c28c-b7a9-4b8e-ad24-32c9139f8c55 admin admin] PATH_INFO: `/v3/projects/4b753dad8a354b8895937eb7c40b935f/users/dd60cb83752d48cb8275d7efd2060e9e/roles/8feebc60363c4feb870718b1721e5324` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:34.812772 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e83c28c-b7a9-4b8e-ad24-32c9139f8c55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b753dad8a354b8895937eb7c40b935f, user_id=dd60cb83752d48cb8275d7efd2060e9e, role_id=8feebc60363c4feb870718b1721e5324)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:34.815802 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e83c28c-b7a9-4b8e-ad24-32c9139f8c55 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:34.825109 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:34.875132 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34a4e58-1df1-42b7-b1ed-ff2481a86892 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:34.876889 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-a34a4e58-1df1-42b7-b1ed-ff2481a86892 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:34.890598 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:34.891250 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34a4e58-1df1-42b7-b1ed-ff2481a86892 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:34.898764 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1284/2604] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:32:34 2022] PUT /identity/v3/projects/4b753dad8a354b8895937eb7c40b935f/users/dd60cb83752d48cb8275d7efd2060e9e/roles/8feebc60363c4feb870718b1721e5324 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:32:34.913628 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ee99bc-0738-439b-bdcc-a1815226400f None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:34.913948 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ee99bc-0738-439b-bdcc-a1815226400f None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:34.914187 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ee99bc-0738-439b-bdcc-a1815226400f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:34.957748 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-e1ee99bc-0738-439b-bdcc-a1815226400f None None] MFA Rules not processed for user `dd60cb83752d48cb8275d7efd2060e9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:32:34.957748 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-e1ee99bc-0738-439b-bdcc-a1815226400f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:34.977517 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7e6252-f88e-4a4c-a71c-bd4c64650589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:34.977517 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7e6252-f88e-4a4c-a71c-bd4c64650589 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:34.977517 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7e6252-f88e-4a4c-a71c-bd4c64650589 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:34.977517 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7e6252-f88e-4a4c-a71c-bd4c64650589 admin admin] PATH_INFO: `/v3/users/afb40c7e64814d3aa37decc43470a4dd` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:34.977912 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7e6252-f88e-4a4c-a71c-bd4c64650589 admin admin] RBAC: Authorizing `identity:delete_user(user_id=afb40c7e64814d3aa37decc43470a4dd)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:34.977912 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7e6252-f88e-4a4c-a71c-bd4c64650589 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:35.019459 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1285/2605] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:32:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:32:35.024223 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ee99bc-0738-439b-bdcc-a1815226400f None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:35.035500 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ee99bc-0738-439b-bdcc-a1815226400f None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:35.043896 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f017ca-b5da-4120-b508-4745523e099c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:35.043896 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f017ca-b5da-4120-b508-4745523e099c admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:35.043896 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f017ca-b5da-4120-b508-4745523e099c admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:35.043896 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f017ca-b5da-4120-b508-4745523e099c admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:35.044391 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f017ca-b5da-4120-b508-4745523e099c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:35.045145 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f017ca-b5da-4120-b508-4745523e099c admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:35.178579 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1286/2606] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:32:35 2022] POST /identity/v3/projects => generated 375 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:32:35.184064 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f017ca-b5da-4120-b508-4745523e099c admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:35.225223 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-fb7e6252-f88e-4a4c-a71c-bd4c64650589 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afb40c7e64814d3aa37decc43470a4dd'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:35.239134 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-fb7e6252-f88e-4a4c-a71c-bd4c64650589 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afb40c7e64814d3aa37decc43470a4dd'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:35.241773 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-fb7e6252-f88e-4a4c-a71c-bd4c64650589 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afb40c7e64814d3aa37decc43470a4dd'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:35.242478 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-fb7e6252-f88e-4a4c-a71c-bd4c64650589 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afb40c7e64814d3aa37decc43470a4dd'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:35.254037 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f017ca-b5da-4120-b508-4745523e099c admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:35.260268 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd3602d-d290-46c5-ad2c-14b3f6e416ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:35.261248 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd3602d-d290-46c5-ad2c-14b3f6e416ff admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:35.261248 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd3602d-d290-46c5-ad2c-14b3f6e416ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:35.261699 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd3602d-d290-46c5-ad2c-14b3f6e416ff admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:35.262326 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd3602d-d290-46c5-ad2c-14b3f6e416ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:35.264317 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd3602d-d290-46c5-ad2c-14b3f6e416ff admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:35.304142 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-fb7e6252-f88e-4a4c-a71c-bd4c64650589 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afb40c7e64814d3aa37decc43470a4dd'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:35.318781 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1321/2607] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:32:34 2022] DELETE /identity/v3/users/afb40c7e64814d3aa37decc43470a4dd => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:32:35.323887 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c76103-436a-4400-9262-d37a6004493d None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:35.324390 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c76103-436a-4400-9262-d37a6004493d None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:35.324490 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c76103-436a-4400-9262-d37a6004493d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:35.344931 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-a7c76103-436a-4400-9262-d37a6004493d None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:32:35.346174 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-a7c76103-436a-4400-9262-d37a6004493d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:35.382162 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1287/2608] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:32:35 2022] POST /identity/v3/users => generated 294 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:32:35.384692 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:35.388485 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd3602d-d290-46c5-ad2c-14b3f6e416ff admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:35.390478 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-8bd3602d-d290-46c5-ad2c-14b3f6e416ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:35.404005 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd3602d-d290-46c5-ad2c-14b3f6e416ff admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:35.410069 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1322/2609] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:32:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:32:35.417957 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44bf920-81bd-4b15-8cea-17e573bf22ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:35.417957 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44bf920-81bd-4b15-8cea-17e573bf22ac admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:35.417957 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44bf920-81bd-4b15-8cea-17e573bf22ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:35.417957 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44bf920-81bd-4b15-8cea-17e573bf22ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:35.418429 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44bf920-81bd-4b15-8cea-17e573bf22ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:35.418429 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44bf920-81bd-4b15-8cea-17e573bf22ac admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:35.420011 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c76103-436a-4400-9262-d37a6004493d None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:35.421410 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-a7c76103-436a-4400-9262-d37a6004493d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:35.429127 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1288/2610] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:32:35 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:32:35.433061 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44bf920-81bd-4b15-8cea-17e573bf22ac admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:35.448534 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44bf920-81bd-4b15-8cea-17e573bf22ac admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:35.455127 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89d327c-616f-46eb-9970-39da05ffc981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:35.456599 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89d327c-616f-46eb-9970-39da05ffc981 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:35.456874 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89d327c-616f-46eb-9970-39da05ffc981 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:35.457150 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89d327c-616f-46eb-9970-39da05ffc981 admin admin] PATH_INFO: `/v3/projects/e6a2465c3cb54aaab40f5295e1f54ef2/users/26fff62009e34be6a5da542e4237de7b/roles/a432dd4735944ecfab2fa5ce1705d356` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:35.476184 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89d327c-616f-46eb-9970-39da05ffc981 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6a2465c3cb54aaab40f5295e1f54ef2, user_id=26fff62009e34be6a5da542e4237de7b, role_id=a432dd4735944ecfab2fa5ce1705d356)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:35.478978 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89d327c-616f-46eb-9970-39da05ffc981 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:35.494615 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c76103-436a-4400-9262-d37a6004493d None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:35.498354 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c36360-42cf-4094-b1cc-00357347818f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:35.498974 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c36360-42cf-4094-b1cc-00357347818f service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:35.499252 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c36360-42cf-4094-b1cc-00357347818f service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:35.499549 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c36360-42cf-4094-b1cc-00357347818f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:35.509310 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c36360-42cf-4094-b1cc-00357347818f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:35.510730 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c36360-42cf-4094-b1cc-00357347818f service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:35.520549 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1323/2611] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:32:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:32:35.581298 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:35.593498 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1289/2612] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:32:35 2022] PUT /identity/v3/projects/e6a2465c3cb54aaab40f5295e1f54ef2/users/26fff62009e34be6a5da542e4237de7b/roles/a432dd4735944ecfab2fa5ce1705d356 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:32:35.597767 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c36360-42cf-4094-b1cc-00357347818f service neutron] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:35.612968 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c36360-42cf-4094-b1cc-00357347818f service neutron] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:35.671820 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8810b3b0-1419-4166-8b3b-69b7a88bf91d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:35.672448 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8810b3b0-1419-4166-8b3b-69b7a88bf91d admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:35.672884 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8810b3b0-1419-4166-8b3b-69b7a88bf91d admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:35.672977 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8810b3b0-1419-4166-8b3b-69b7a88bf91d admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:35.673653 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8810b3b0-1419-4166-8b3b-69b7a88bf91d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:35.676171 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8810b3b0-1419-4166-8b3b-69b7a88bf91d admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:35.694704 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1324/2613] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:32:35 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:32:35.696389 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89d327c-616f-46eb-9970-39da05ffc981 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:35.706847 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89d327c-616f-46eb-9970-39da05ffc981 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:35.711842 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adc22f5-4da7-4d04-b32c-6c33833853e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:35.712623 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adc22f5-4da7-4d04-b32c-6c33833853e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:35.712968 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adc22f5-4da7-4d04-b32c-6c33833853e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:35.713312 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adc22f5-4da7-4d04-b32c-6c33833853e7 admin admin] PATH_INFO: `/v3/projects/e6a2465c3cb54aaab40f5295e1f54ef2/users/26fff62009e34be6a5da542e4237de7b/roles/a432dd4735944ecfab2fa5ce1705d356` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:35.715354 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adc22f5-4da7-4d04-b32c-6c33833853e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6a2465c3cb54aaab40f5295e1f54ef2, user_id=26fff62009e34be6a5da542e4237de7b, role_id=a432dd4735944ecfab2fa5ce1705d356)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:35.718095 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adc22f5-4da7-4d04-b32c-6c33833853e7 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:35.732106 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1290/2614] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:32:35 2022] PUT /identity/v3/projects/e6a2465c3cb54aaab40f5295e1f54ef2/users/26fff62009e34be6a5da542e4237de7b/roles/a432dd4735944ecfab2fa5ce1705d356 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:32:35.745093 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e590ee4b-fbd7-42ac-af8d-df17f594a4b3 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:35.745518 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e590ee4b-fbd7-42ac-af8d-df17f594a4b3 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:35.745850 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e590ee4b-fbd7-42ac-af8d-df17f594a4b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:35.786977 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-e590ee4b-fbd7-42ac-af8d-df17f594a4b3 None None] MFA Rules not processed for user `26fff62009e34be6a5da542e4237de7b`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:32:35.789094 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-e590ee4b-fbd7-42ac-af8d-df17f594a4b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:35.868965 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1325/2615] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:32:35 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:32:35.880558 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adc22f5-4da7-4d04-b32c-6c33833853e7 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:35.882375 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-0adc22f5-4da7-4d04-b32c-6c33833853e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:35.943073 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adc22f5-4da7-4d04-b32c-6c33833853e7 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:35.947048 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2876fbb7-a9d4-4648-beda-653ba1a88d6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e31f2728056b460fa474478eb85ca1d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:35.947708 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2876fbb7-a9d4-4648-beda-653ba1a88d6d service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:35.947992 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2876fbb7-a9d4-4648-beda-653ba1a88d6d service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:35.948261 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2876fbb7-a9d4-4648-beda-653ba1a88d6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:35.950008 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-2876fbb7-a9d4-4648-beda-653ba1a88d6d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:35.995095 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e590ee4b-fbd7-42ac-af8d-df17f594a4b3 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:35.999248 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-e590ee4b-fbd7-42ac-af8d-df17f594a4b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:36.013830 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2876fbb7-a9d4-4648-beda-653ba1a88d6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:36.015900 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2876fbb7-a9d4-4648-beda-653ba1a88d6d service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:36.016873 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:36.034979 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1291/2616] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:32:35 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:32:36.100825 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:INSERT=4,keystone:DELETE=8,keystone:SELECT=88 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:36.114888 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e590ee4b-fbd7-42ac-af8d-df17f594a4b3 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:36.119980 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074eb164-a4bc-4422-8f1f-d3a3a722cb13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:36.120620 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074eb164-a4bc-4422-8f1f-d3a3a722cb13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:36.120900 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074eb164-a4bc-4422-8f1f-d3a3a722cb13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:36.121173 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074eb164-a4bc-4422-8f1f-d3a3a722cb13 admin admin] PATH_INFO: `/v3/projects/4a4d68ac121e4a61a9f7819f05c78d56` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:36.122526 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074eb164-a4bc-4422-8f1f-d3a3a722cb13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a4d68ac121e4a61a9f7819f05c78d56)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:36.124974 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074eb164-a4bc-4422-8f1f-d3a3a722cb13 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:36.269532 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-074eb164-a4bc-4422-8f1f-d3a3a722cb13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a4d68ac121e4a61a9f7819f05c78d56', 'request_id': 'req-074eb164-a4bc-4422-8f1f-d3a3a722cb13'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:36.275610 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-074eb164-a4bc-4422-8f1f-d3a3a722cb13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a4d68ac121e4a61a9f7819f05c78d56', 'request_id': 'req-074eb164-a4bc-4422-8f1f-d3a3a722cb13'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:36.372908 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-074eb164-a4bc-4422-8f1f-d3a3a722cb13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a4d68ac121e4a61a9f7819f05c78d56', 'request_id': 'req-074eb164-a4bc-4422-8f1f-d3a3a722cb13'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:36.385636 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-074eb164-a4bc-4422-8f1f-d3a3a722cb13 admin admin] The token cache is being invalidate because project 4a4d68ac121e4a61a9f7819f05c78d56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:32:36.385930 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-074eb164-a4bc-4422-8f1f-d3a3a722cb13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:36.386594 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-074eb164-a4bc-4422-8f1f-d3a3a722cb13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:36.388167 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1326/2617] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:32:35 2022] DELETE /identity/v3/projects/4a4d68ac121e4a61a9f7819f05c78d56 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:32:36.392203 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2876fbb7-a9d4-4648-beda-653ba1a88d6d service cinder] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:36.404232 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2876fbb7-a9d4-4648-beda-653ba1a88d6d service cinder] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:36.463547 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e2549f-739c-4c3c-9142-27dc7aeda6dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:36.464268 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e2549f-739c-4c3c-9142-27dc7aeda6dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:36.464546 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e2549f-739c-4c3c-9142-27dc7aeda6dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:36.464830 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e2549f-739c-4c3c-9142-27dc7aeda6dc admin admin] PATH_INFO: `/v3/users/f765a959908e4992ad41ec66d21577f7` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:36.466159 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e2549f-739c-4c3c-9142-27dc7aeda6dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=f765a959908e4992ad41ec66d21577f7)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:36.468162 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e2549f-739c-4c3c-9142-27dc7aeda6dc admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:36.686365 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-42e2549f-739c-4c3c-9142-27dc7aeda6dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f765a959908e4992ad41ec66d21577f7'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:36.699748 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-42e2549f-739c-4c3c-9142-27dc7aeda6dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f765a959908e4992ad41ec66d21577f7'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:36.700600 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-42e2549f-739c-4c3c-9142-27dc7aeda6dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f765a959908e4992ad41ec66d21577f7'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:36.709112 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-42e2549f-739c-4c3c-9142-27dc7aeda6dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f765a959908e4992ad41ec66d21577f7'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:36.709823 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-42e2549f-739c-4c3c-9142-27dc7aeda6dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f765a959908e4992ad41ec66d21577f7'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:36.770500 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:36.804052 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1292/2618] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:32:36 2022] DELETE /identity/v3/users/f765a959908e4992ad41ec66d21577f7 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:32:36.814102 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:37.079524 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:37.172764 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:37.336037 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074eb164-a4bc-4422-8f1f-d3a3a722cb13 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:37.337527 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-074eb164-a4bc-4422-8f1f-d3a3a722cb13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:37.356212 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074eb164-a4bc-4422-8f1f-d3a3a722cb13 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:37.416433 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35c2ee3-ae0d-4a46-90a2-a517341ae416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:37.417147 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35c2ee3-ae0d-4a46-90a2-a517341ae416 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:37.417465 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35c2ee3-ae0d-4a46-90a2-a517341ae416 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:37.417746 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35c2ee3-ae0d-4a46-90a2-a517341ae416 admin admin] PATH_INFO: `/v3/projects/405f9d7d02c642fc822ffef7959ab0ca` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:37.418989 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35c2ee3-ae0d-4a46-90a2-a517341ae416 admin admin] RBAC: Authorizing `identity:delete_project(project_id=405f9d7d02c642fc822ffef7959ab0ca)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:37.421030 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35c2ee3-ae0d-4a46-90a2-a517341ae416 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:37.636916 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-c35c2ee3-ae0d-4a46-90a2-a517341ae416 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '405f9d7d02c642fc822ffef7959ab0ca', 'request_id': 'req-c35c2ee3-ae0d-4a46-90a2-a517341ae416'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:37.642321 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-c35c2ee3-ae0d-4a46-90a2-a517341ae416 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '405f9d7d02c642fc822ffef7959ab0ca', 'request_id': 'req-c35c2ee3-ae0d-4a46-90a2-a517341ae416'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:37.750272 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-c35c2ee3-ae0d-4a46-90a2-a517341ae416 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '405f9d7d02c642fc822ffef7959ab0ca', 'request_id': 'req-c35c2ee3-ae0d-4a46-90a2-a517341ae416'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:37.763169 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-c35c2ee3-ae0d-4a46-90a2-a517341ae416 admin admin] The token cache is being invalidate because project 405f9d7d02c642fc822ffef7959ab0ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:32:37.763495 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-c35c2ee3-ae0d-4a46-90a2-a517341ae416 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:37.764127 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-c35c2ee3-ae0d-4a46-90a2-a517341ae416 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:37.765643 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1327/2619] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:32:37 2022] DELETE /identity/v3/projects/405f9d7d02c642fc822ffef7959ab0ca => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:32:37.772998 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e69ea9b-581a-4478-85a6-b8df9de3faec None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:37.773529 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e69ea9b-581a-4478-85a6-b8df9de3faec None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:37.773529 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e69ea9b-581a-4478-85a6-b8df9de3faec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:37.795109 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-4e69ea9b-581a-4478-85a6-b8df9de3faec None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:32:37.796342 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-4e69ea9b-581a-4478-85a6-b8df9de3faec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:37.873382 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1293/2620] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:32:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:32:37.878577 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:37.882358 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4d1eaf-68aa-4abf-8de9-09d6020bf61d None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:37.882642 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4d1eaf-68aa-4abf-8de9-09d6020bf61d None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:37.882917 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4d1eaf-68aa-4abf-8de9-09d6020bf61d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:37.906970 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-aa4d1eaf-68aa-4abf-8de9-09d6020bf61d None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:32:37.908020 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-aa4d1eaf-68aa-4abf-8de9-09d6020bf61d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:37.920511 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1328/2621] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:32:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:32:37.922452 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e69ea9b-581a-4478-85a6-b8df9de3faec None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:37.933435 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e69ea9b-581a-4478-85a6-b8df9de3faec None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:37.941576 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5ff1b3-ad32-4dfd-ab64-1857d13247ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:37.941576 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5ff1b3-ad32-4dfd-ab64-1857d13247ce admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:37.941576 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5ff1b3-ad32-4dfd-ab64-1857d13247ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:37.941576 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5ff1b3-ad32-4dfd-ab64-1857d13247ce admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:37.942005 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5ff1b3-ad32-4dfd-ab64-1857d13247ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:32:37.942005 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5ff1b3-ad32-4dfd-ab64-1857d13247ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:37.942619 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5ff1b3-ad32-4dfd-ab64-1857d13247ce admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:37.954637 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1294/2622] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Aug 4 21:32:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:32:37.959143 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4d1eaf-68aa-4abf-8de9-09d6020bf61d None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:37.965748 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:37.975975 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4d1eaf-68aa-4abf-8de9-09d6020bf61d None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:37.976486 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139897f6-4a95-4271-98ff-d541ca8decef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:37.977110 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139897f6-4a95-4271-98ff-d541ca8decef admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:37.977394 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139897f6-4a95-4271-98ff-d541ca8decef admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:37.977669 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139897f6-4a95-4271-98ff-d541ca8decef admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:37.978348 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139897f6-4a95-4271-98ff-d541ca8decef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:37.980299 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139897f6-4a95-4271-98ff-d541ca8decef admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:38.086598 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1329/2623] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:32:37 2022] POST /identity/v3/projects => generated 377 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:32:38.089813 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5ff1b3-ad32-4dfd-ab64-1857d13247ce admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:38.101346 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5ff1b3-ad32-4dfd-ab64-1857d13247ce admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:38.169357 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79470f83-87d0-44c7-a4ef-556fecc6875d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:38.170102 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79470f83-87d0-44c7-a4ef-556fecc6875d admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:38.170703 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79470f83-87d0-44c7-a4ef-556fecc6875d admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:38.171186 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79470f83-87d0-44c7-a4ef-556fecc6875d admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:38.172007 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79470f83-87d0-44c7-a4ef-556fecc6875d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:38.175557 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79470f83-87d0-44c7-a4ef-556fecc6875d admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:38.291180 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1295/2624] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:32:38 2022] POST /identity/v3/users => generated 289 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:32:38.296627 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139897f6-4a95-4271-98ff-d541ca8decef admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:38.307830 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139897f6-4a95-4271-98ff-d541ca8decef admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:38.314420 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da349688-6ada-4f99-9fea-a640f8aff939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:38.315147 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da349688-6ada-4f99-9fea-a640f8aff939 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:38.315504 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da349688-6ada-4f99-9fea-a640f8aff939 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:38.315830 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da349688-6ada-4f99-9fea-a640f8aff939 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:38.316621 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da349688-6ada-4f99-9fea-a640f8aff939 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:38.318228 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da349688-6ada-4f99-9fea-a640f8aff939 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:38.338365 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1330/2625] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:32:38 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:32:38.342667 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79470f83-87d0-44c7-a4ef-556fecc6875d admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:38.358879 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79470f83-87d0-44c7-a4ef-556fecc6875d admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:38.364853 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd2d576-e913-4d45-a612-08272a6d2d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:38.365469 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd2d576-e913-4d45-a612-08272a6d2d40 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:38.365752 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd2d576-e913-4d45-a612-08272a6d2d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:38.366027 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd2d576-e913-4d45-a612-08272a6d2d40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:38.366731 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd2d576-e913-4d45-a612-08272a6d2d40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:38.368441 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd2d576-e913-4d45-a612-08272a6d2d40 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:38.380959 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1296/2626] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:32:38 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:32:38.389860 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da349688-6ada-4f99-9fea-a640f8aff939 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:38.400310 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da349688-6ada-4f99-9fea-a640f8aff939 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:38.405134 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83aae8a-6ebe-480d-b141-c56b4fa9b846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:38.405827 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83aae8a-6ebe-480d-b141-c56b4fa9b846 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:38.406112 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83aae8a-6ebe-480d-b141-c56b4fa9b846 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:38.406452 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83aae8a-6ebe-480d-b141-c56b4fa9b846 admin admin] PATH_INFO: `/v3/projects/66698488e28e43a6b3b3f78abe6cec16/users/08e724743bb94e34a1c8a786097499cb/roles/8feebc60363c4feb870718b1721e5324` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:38.422129 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83aae8a-6ebe-480d-b141-c56b4fa9b846 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66698488e28e43a6b3b3f78abe6cec16, user_id=08e724743bb94e34a1c8a786097499cb, role_id=8feebc60363c4feb870718b1721e5324)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:38.425505 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83aae8a-6ebe-480d-b141-c56b4fa9b846 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:38.529962 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1331/2627] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:32:38 2022] PUT /identity/v3/projects/66698488e28e43a6b3b3f78abe6cec16/users/08e724743bb94e34a1c8a786097499cb/roles/8feebc60363c4feb870718b1721e5324 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:32:38.541523 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7676201f-fb0c-4190-b043-d70580eab6c2 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:38.541918 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7676201f-fb0c-4190-b043-d70580eab6c2 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:38.542350 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7676201f-fb0c-4190-b043-d70580eab6c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:38.572480 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-7676201f-fb0c-4190-b043-d70580eab6c2 None None] MFA Rules not processed for user `08e724743bb94e34a1c8a786097499cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:32:38.573685 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-7676201f-fb0c-4190-b043-d70580eab6c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:38.623452 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:38.644163 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1297/2628] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:32:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:32:38.656267 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83aae8a-6ebe-480d-b141-c56b4fa9b846 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:38.659624 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-d83aae8a-6ebe-480d-b141-c56b4fa9b846 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:38.727283 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83aae8a-6ebe-480d-b141-c56b4fa9b846 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:38.731398 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3137359-002f-4366-99ae-18a7b092284b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e31f2728056b460fa474478eb85ca1d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:38.732657 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3137359-002f-4366-99ae-18a7b092284b service cinder] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:38.733043 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3137359-002f-4366-99ae-18a7b092284b service cinder] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:38.733223 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3137359-002f-4366-99ae-18a7b092284b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:38.743646 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3137359-002f-4366-99ae-18a7b092284b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:38.745161 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3137359-002f-4366-99ae-18a7b092284b service cinder] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:38.753945 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1332/2629] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:32:38 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:32:38.850712 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7676201f-fb0c-4190-b043-d70580eab6c2 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:38.852281 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-7676201f-fb0c-4190-b043-d70580eab6c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:38.927063 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7676201f-fb0c-4190-b043-d70580eab6c2 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:38.932618 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0db5e9-9ce8-46a7-b51c-6b4f08f89035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:38.933258 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0db5e9-9ce8-46a7-b51c-6b4f08f89035 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:38.933547 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0db5e9-9ce8-46a7-b51c-6b4f08f89035 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:38.933825 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0db5e9-9ce8-46a7-b51c-6b4f08f89035 admin admin] PATH_INFO: `/v3/users/f6ef1a44226c4ca183d189147cb78413` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:38.934975 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0db5e9-9ce8-46a7-b51c-6b4f08f89035 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6ef1a44226c4ca183d189147cb78413)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:38.937031 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0db5e9-9ce8-46a7-b51c-6b4f08f89035 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:39.141702 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-aa0db5e9-9ce8-46a7-b51c-6b4f08f89035 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6ef1a44226c4ca183d189147cb78413'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:39.154813 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-aa0db5e9-9ce8-46a7-b51c-6b4f08f89035 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6ef1a44226c4ca183d189147cb78413'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:39.155549 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-aa0db5e9-9ce8-46a7-b51c-6b4f08f89035 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6ef1a44226c4ca183d189147cb78413'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:39.163773 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-aa0db5e9-9ce8-46a7-b51c-6b4f08f89035 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6ef1a44226c4ca183d189147cb78413'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:39.164439 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-aa0db5e9-9ce8-46a7-b51c-6b4f08f89035 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6ef1a44226c4ca183d189147cb78413'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:39.266230 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1298/2630] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:32:38 2022] DELETE /identity/v3/users/f6ef1a44226c4ca183d189147cb78413 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:32:39.274397 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c831d29c-4595-45e5-932a-cc6e30f0eb8e None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:39.274884 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c831d29c-4595-45e5-932a-cc6e30f0eb8e None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:39.275027 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c831d29c-4595-45e5-932a-cc6e30f0eb8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:39.296714 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-c831d29c-4595-45e5-932a-cc6e30f0eb8e None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:32:39.299477 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-c831d29c-4595-45e5-932a-cc6e30f0eb8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:39.336315 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:39.363463 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1333/2631] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:32:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:32:39.379114 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0db5e9-9ce8-46a7-b51c-6b4f08f89035 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:39.380600 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-aa0db5e9-9ce8-46a7-b51c-6b4f08f89035 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:39.451158 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0db5e9-9ce8-46a7-b51c-6b4f08f89035 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:39.454880 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b079d7e-26f5-4224-bf1e-8fea7965ea8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:39.456087 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b079d7e-26f5-4224-bf1e-8fea7965ea8e service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:39.456373 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b079d7e-26f5-4224-bf1e-8fea7965ea8e service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:39.456791 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b079d7e-26f5-4224-bf1e-8fea7965ea8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:39.469939 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b079d7e-26f5-4224-bf1e-8fea7965ea8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:39.469939 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b079d7e-26f5-4224-bf1e-8fea7965ea8e service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:39.478596 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1299/2632] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:32:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:32:39.965773 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c831d29c-4595-45e5-932a-cc6e30f0eb8e None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:39.965773 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-c831d29c-4595-45e5-932a-cc6e30f0eb8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:39.977326 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c831d29c-4595-45e5-932a-cc6e30f0eb8e None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:39.982477 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361517d6-64df-43e2-8153-2d5833a14541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:39.983177 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361517d6-64df-43e2-8153-2d5833a14541 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:39.983547 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361517d6-64df-43e2-8153-2d5833a14541 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:39.983896 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361517d6-64df-43e2-8153-2d5833a14541 admin admin] PATH_INFO: `/v3/projects/3188dcfd3c944120bf5e34323cc4cc6d` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:39.984982 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361517d6-64df-43e2-8153-2d5833a14541 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3188dcfd3c944120bf5e34323cc4cc6d)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:39.986945 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361517d6-64df-43e2-8153-2d5833a14541 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:40.130812 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-361517d6-64df-43e2-8153-2d5833a14541 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3188dcfd3c944120bf5e34323cc4cc6d', 'request_id': 'req-361517d6-64df-43e2-8153-2d5833a14541'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:40.136445 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-361517d6-64df-43e2-8153-2d5833a14541 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3188dcfd3c944120bf5e34323cc4cc6d', 'request_id': 'req-361517d6-64df-43e2-8153-2d5833a14541'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:40.214822 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-361517d6-64df-43e2-8153-2d5833a14541 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3188dcfd3c944120bf5e34323cc4cc6d', 'request_id': 'req-361517d6-64df-43e2-8153-2d5833a14541'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:40.227453 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-361517d6-64df-43e2-8153-2d5833a14541 admin admin] The token cache is being invalidate because project 3188dcfd3c944120bf5e34323cc4cc6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:32:40.227748 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-361517d6-64df-43e2-8153-2d5833a14541 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:40.231608 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-361517d6-64df-43e2-8153-2d5833a14541 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:40.231608 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1334/2633] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:32:39 2022] DELETE /identity/v3/projects/3188dcfd3c944120bf5e34323cc4cc6d => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:32:40.234201 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b079d7e-26f5-4224-bf1e-8fea7965ea8e service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:40.244655 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b079d7e-26f5-4224-bf1e-8fea7965ea8e service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:40.255106 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:40.307060 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3074ec52-25ff-4534-a50f-2862c33ad9ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:40.307795 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3074ec52-25ff-4534-a50f-2862c33ad9ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:40.308080 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3074ec52-25ff-4534-a50f-2862c33ad9ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:40.308355 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3074ec52-25ff-4534-a50f-2862c33ad9ea admin admin] PATH_INFO: `/v3/users/154085a5ae3c41e5afb50090e6099851` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:40.309518 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3074ec52-25ff-4534-a50f-2862c33ad9ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=154085a5ae3c41e5afb50090e6099851)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:40.311538 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3074ec52-25ff-4534-a50f-2862c33ad9ea admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:40.555134 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-3074ec52-25ff-4534-a50f-2862c33ad9ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '154085a5ae3c41e5afb50090e6099851'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:40.568818 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-3074ec52-25ff-4534-a50f-2862c33ad9ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '154085a5ae3c41e5afb50090e6099851'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:40.569549 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-3074ec52-25ff-4534-a50f-2862c33ad9ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '154085a5ae3c41e5afb50090e6099851'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:40.577005 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-3074ec52-25ff-4534-a50f-2862c33ad9ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '154085a5ae3c41e5afb50090e6099851'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:40.577627 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-3074ec52-25ff-4534-a50f-2862c33ad9ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '154085a5ae3c41e5afb50090e6099851'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:40.683916 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1300/2634] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:32:40 2022] DELETE /identity/v3/users/154085a5ae3c41e5afb50090e6099851 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:32:41.100418 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361517d6-64df-43e2-8153-2d5833a14541 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:41.101925 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-361517d6-64df-43e2-8153-2d5833a14541 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:41.118659 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361517d6-64df-43e2-8153-2d5833a14541 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:41.179662 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3bdc22-fea0-4ae2-b2df-b33fb3d5bb7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:41.180502 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3bdc22-fea0-4ae2-b2df-b33fb3d5bb7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:41.180824 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3bdc22-fea0-4ae2-b2df-b33fb3d5bb7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:41.181150 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3bdc22-fea0-4ae2-b2df-b33fb3d5bb7d admin admin] PATH_INFO: `/v3/projects/f289f5bb83a04b619ca44b03ad23bf26` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:41.182460 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3bdc22-fea0-4ae2-b2df-b33fb3d5bb7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f289f5bb83a04b619ca44b03ad23bf26)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:41.184620 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3bdc22-fea0-4ae2-b2df-b33fb3d5bb7d admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:41.282371 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-7d3bdc22-fea0-4ae2-b2df-b33fb3d5bb7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f289f5bb83a04b619ca44b03ad23bf26', 'request_id': 'req-7d3bdc22-fea0-4ae2-b2df-b33fb3d5bb7d'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:41.288011 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-7d3bdc22-fea0-4ae2-b2df-b33fb3d5bb7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f289f5bb83a04b619ca44b03ad23bf26', 'request_id': 'req-7d3bdc22-fea0-4ae2-b2df-b33fb3d5bb7d'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:41.380605 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-7d3bdc22-fea0-4ae2-b2df-b33fb3d5bb7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f289f5bb83a04b619ca44b03ad23bf26', 'request_id': 'req-7d3bdc22-fea0-4ae2-b2df-b33fb3d5bb7d'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:41.382023 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:41.392555 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-7d3bdc22-fea0-4ae2-b2df-b33fb3d5bb7d admin admin] The token cache is being invalidate because project f289f5bb83a04b619ca44b03ad23bf26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:32:41.392846 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-7d3bdc22-fea0-4ae2-b2df-b33fb3d5bb7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:41.394134 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-7d3bdc22-fea0-4ae2-b2df-b33fb3d5bb7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:32:41.395714 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1335/2635] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:32:41 2022] DELETE /identity/v3/projects/f289f5bb83a04b619ca44b03ad23bf26 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:32:41.400531 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cc7dae-be26-4443-9666-5443b330708f None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:41.401017 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cc7dae-be26-4443-9666-5443b330708f None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:41.401119 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cc7dae-be26-4443-9666-5443b330708f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:41.420089 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-c4cc7dae-be26-4443-9666-5443b330708f None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:32:41.421270 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-c4cc7dae-be26-4443-9666-5443b330708f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:41.483885 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1301/2636] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:32:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:32:41.492428 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa611f-dfbf-465a-acc7-ef78de2dd975 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:41.492720 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa611f-dfbf-465a-acc7-ef78de2dd975 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:41.492994 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa611f-dfbf-465a-acc7-ef78de2dd975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:41.515572 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-c3fa611f-dfbf-465a-acc7-ef78de2dd975 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:32:41.516805 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-c3fa611f-dfbf-465a-acc7-ef78de2dd975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:41.524936 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1336/2637] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:32:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:32:41.529223 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cc7dae-be26-4443-9666-5443b330708f None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:41.538993 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cc7dae-be26-4443-9666-5443b330708f None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:41.543776 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e66b767-3e2d-4d9f-964e-f9ec1f0f0e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:41.544395 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e66b767-3e2d-4d9f-964e-f9ec1f0f0e3a admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:41.544678 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e66b767-3e2d-4d9f-964e-f9ec1f0f0e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:41.544952 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e66b767-3e2d-4d9f-964e-f9ec1f0f0e3a admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:41.545654 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e66b767-3e2d-4d9f-964e-f9ec1f0f0e3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:32:41.545963 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e66b767-3e2d-4d9f-964e-f9ec1f0f0e3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:41.547606 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e66b767-3e2d-4d9f-964e-f9ec1f0f0e3a admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:41.566264 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1302/2638] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Aug 4 21:32:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:32:41.567578 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fa611f-dfbf-465a-acc7-ef78de2dd975 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:41.577737 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fa611f-dfbf-465a-acc7-ef78de2dd975 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:41.583069 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52123a10-98a0-4985-a3c3-d8000b8ee0ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:41.583717 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52123a10-98a0-4985-a3c3-d8000b8ee0ee admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:41.584030 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52123a10-98a0-4985-a3c3-d8000b8ee0ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:41.584318 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52123a10-98a0-4985-a3c3-d8000b8ee0ee admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:41.585063 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52123a10-98a0-4985-a3c3-d8000b8ee0ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:41.587087 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52123a10-98a0-4985-a3c3-d8000b8ee0ee admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:41.687258 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1337/2639] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:32:41 2022] POST /identity/v3/projects => generated 385 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:32:41.690101 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e66b767-3e2d-4d9f-964e-f9ec1f0f0e3a admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:41.701417 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e66b767-3e2d-4d9f-964e-f9ec1f0f0e3a admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:41.718876 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:41.760539 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38a957f-ef66-4e82-bae3-ff2b5bae7d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:41.761274 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38a957f-ef66-4e82-bae3-ff2b5bae7d5e admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:41.761600 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38a957f-ef66-4e82-bae3-ff2b5bae7d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:41.761946 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38a957f-ef66-4e82-bae3-ff2b5bae7d5e admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:41.762663 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38a957f-ef66-4e82-bae3-ff2b5bae7d5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:41.764711 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38a957f-ef66-4e82-bae3-ff2b5bae7d5e admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:41.870686 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1303/2640] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:32:41 2022] POST /identity/v3/users => generated 293 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:32:41.875171 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52123a10-98a0-4985-a3c3-d8000b8ee0ee admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:41.886880 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52123a10-98a0-4985-a3c3-d8000b8ee0ee admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:41.892966 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd02a47b-592b-49cb-aa66-6d504eb6ba52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:41.893730 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd02a47b-592b-49cb-aa66-6d504eb6ba52 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:41.893998 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd02a47b-592b-49cb-aa66-6d504eb6ba52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:41.894295 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd02a47b-592b-49cb-aa66-6d504eb6ba52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:41.895072 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd02a47b-592b-49cb-aa66-6d504eb6ba52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:41.896783 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd02a47b-592b-49cb-aa66-6d504eb6ba52 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:41.907884 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1338/2641] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:32:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:32:41.916684 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38a957f-ef66-4e82-bae3-ff2b5bae7d5e admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:41.927307 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38a957f-ef66-4e82-bae3-ff2b5bae7d5e admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:41.931967 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29bdd81-8da2-4031-b32a-1410fa480eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:41.932579 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29bdd81-8da2-4031-b32a-1410fa480eb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:41.932859 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29bdd81-8da2-4031-b32a-1410fa480eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:41.933133 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29bdd81-8da2-4031-b32a-1410fa480eb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:41.933828 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29bdd81-8da2-4031-b32a-1410fa480eb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:41.935464 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29bdd81-8da2-4031-b32a-1410fa480eb7 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:41.949394 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1304/2642] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:32:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:32:41.953592 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd02a47b-592b-49cb-aa66-6d504eb6ba52 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:41.963961 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd02a47b-592b-49cb-aa66-6d504eb6ba52 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:41.968662 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d198a6-05af-4ef4-8387-359fd9ab682b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:41.969329 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d198a6-05af-4ef4-8387-359fd9ab682b admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:41.969617 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d198a6-05af-4ef4-8387-359fd9ab682b admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:41.972134 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d198a6-05af-4ef4-8387-359fd9ab682b admin admin] PATH_INFO: `/v3/projects/07d51f83113a4885a4162bf1f27a2633/users/0f97b8f68c254c6bb60be280a7ec4993/roles/8feebc60363c4feb870718b1721e5324` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:41.986431 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d198a6-05af-4ef4-8387-359fd9ab682b admin admin] RBAC: Authorizing `identity:create_grant(project_id=07d51f83113a4885a4162bf1f27a2633, user_id=0f97b8f68c254c6bb60be280a7ec4993, role_id=8feebc60363c4feb870718b1721e5324)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:41.999591 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d198a6-05af-4ef4-8387-359fd9ab682b admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:42.089832 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1339/2643] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:32:41 2022] PUT /identity/v3/projects/07d51f83113a4885a4162bf1f27a2633/users/0f97b8f68c254c6bb60be280a7ec4993/roles/8feebc60363c4feb870718b1721e5324 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:32:42.103169 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d2556b-4afa-4e1e-b4a7-56fb8d82e6d6 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:42.103775 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d2556b-4afa-4e1e-b4a7-56fb8d82e6d6 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:42.104284 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d2556b-4afa-4e1e-b4a7-56fb8d82e6d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:42.137922 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-40d2556b-4afa-4e1e-b4a7-56fb8d82e6d6 None None] MFA Rules not processed for user `0f97b8f68c254c6bb60be280a7ec4993`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:32:42.139254 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-40d2556b-4afa-4e1e-b4a7-56fb8d82e6d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:42.212784 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1305/2644] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:32:42 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:32:42.217421 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d198a6-05af-4ef4-8387-359fd9ab682b admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:42.297843 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d198a6-05af-4ef4-8387-359fd9ab682b admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:42.304563 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8903d2a-94ec-4dc9-bc63-bd5a4c55922d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:42.305253 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8903d2a-94ec-4dc9-bc63-bd5a4c55922d admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:42.305542 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8903d2a-94ec-4dc9-bc63-bd5a4c55922d admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:42.305822 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8903d2a-94ec-4dc9-bc63-bd5a4c55922d admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:42.306521 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8903d2a-94ec-4dc9-bc63-bd5a4c55922d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:42.308552 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8903d2a-94ec-4dc9-bc63-bd5a4c55922d admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:42.416484 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1340/2645] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:32:42 2022] POST /identity/v3/projects => generated 383 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:32:42.421245 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d2556b-4afa-4e1e-b4a7-56fb8d82e6d6 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:42.434075 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d2556b-4afa-4e1e-b4a7-56fb8d82e6d6 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:42.497427 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fd9e47-bb8f-44fc-a37d-bebf5d5c97a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:42.497705 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fd9e47-bb8f-44fc-a37d-bebf5d5c97a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:42.498039 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fd9e47-bb8f-44fc-a37d-bebf5d5c97a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:42.499584 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fd9e47-bb8f-44fc-a37d-bebf5d5c97a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:42.499584 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fd9e47-bb8f-44fc-a37d-bebf5d5c97a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:42.501176 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fd9e47-bb8f-44fc-a37d-bebf5d5c97a3 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:42.524281 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8903d2a-94ec-4dc9-bc63-bd5a4c55922d admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:42.526636 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-d8903d2a-94ec-4dc9-bc63-bd5a4c55922d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:42.596552 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8903d2a-94ec-4dc9-bc63-bd5a4c55922d admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:42.600295 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a97fb3-7d0c-4f8c-9cdf-fc78ce9baad1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e31f2728056b460fa474478eb85ca1d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:42.600954 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a97fb3-7d0c-4f8c-9cdf-fc78ce9baad1 service cinder] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:42.601233 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a97fb3-7d0c-4f8c-9cdf-fc78ce9baad1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:42.601504 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a97fb3-7d0c-4f8c-9cdf-fc78ce9baad1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:42.603353 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-03a97fb3-7d0c-4f8c-9cdf-fc78ce9baad1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:42.638190 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1306/2646] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:32:42 2022] POST /identity/v3/users => generated 298 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:32:42.642544 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fd9e47-bb8f-44fc-a37d-bebf5d5c97a3 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:42.656538 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fd9e47-bb8f-44fc-a37d-bebf5d5c97a3 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:42.668904 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b516865-d180-4e02-b167-54c5ceb36822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:42.669684 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:42.670559 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b516865-d180-4e02-b167-54c5ceb36822 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:42.670994 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b516865-d180-4e02-b167-54c5ceb36822 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:42.671285 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b516865-d180-4e02-b167-54c5ceb36822 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:42.672670 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b516865-d180-4e02-b167-54c5ceb36822 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:42.675372 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b516865-d180-4e02-b167-54c5ceb36822 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:42.687125 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1307/2647] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:32:42 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:32:42.692541 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b516865-d180-4e02-b167-54c5ceb36822 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:42.704501 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b516865-d180-4e02-b167-54c5ceb36822 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:42.715206 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c70cb11-090e-495a-9806-39a767baaae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:42.715956 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c70cb11-090e-495a-9806-39a767baaae5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:42.716241 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c70cb11-090e-495a-9806-39a767baaae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:42.718795 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c70cb11-090e-495a-9806-39a767baaae5 admin admin] PATH_INFO: `/v3/projects/4bc201b9ec6c4e709a93c22574f29e79/users/f25a75d1055e4f10bee83e35645de45c/roles/a432dd4735944ecfab2fa5ce1705d356` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:42.719133 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:42.723678 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a97fb3-7d0c-4f8c-9cdf-fc78ce9baad1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:42.725299 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a97fb3-7d0c-4f8c-9cdf-fc78ce9baad1 service cinder] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:42.733539 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c70cb11-090e-495a-9806-39a767baaae5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bc201b9ec6c4e709a93c22574f29e79, user_id=f25a75d1055e4f10bee83e35645de45c, role_id=a432dd4735944ecfab2fa5ce1705d356)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:42.736078 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1341/2648] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:32:42 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:32:42.736320 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c70cb11-090e-495a-9806-39a767baaae5 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:42.882807 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1308/2649] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:32:42 2022] PUT /identity/v3/projects/4bc201b9ec6c4e709a93c22574f29e79/users/f25a75d1055e4f10bee83e35645de45c/roles/a432dd4735944ecfab2fa5ce1705d356 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:32:42.887910 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a97fb3-7d0c-4f8c-9cdf-fc78ce9baad1 service cinder] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:42.903249 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a97fb3-7d0c-4f8c-9cdf-fc78ce9baad1 service cinder] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:42.973688 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ff9918-a975-4b17-9580-49ee4d97fb8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:42.974292 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff9918-a975-4b17-9580-49ee4d97fb8c admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:42.974577 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff9918-a975-4b17-9580-49ee4d97fb8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:42.974868 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff9918-a975-4b17-9580-49ee4d97fb8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:42.975599 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ff9918-a975-4b17-9580-49ee4d97fb8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:42.977228 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ff9918-a975-4b17-9580-49ee4d97fb8c admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:42.988742 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1342/2650] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:32:42 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:32:42.993075 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c70cb11-090e-495a-9806-39a767baaae5 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:43.004142 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c70cb11-090e-495a-9806-39a767baaae5 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:43.009017 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6754a96-782e-42ae-8edd-4bd2478d66ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:43.009649 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6754a96-782e-42ae-8edd-4bd2478d66ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:43.010141 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6754a96-782e-42ae-8edd-4bd2478d66ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:43.010141 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6754a96-782e-42ae-8edd-4bd2478d66ab admin admin] PATH_INFO: `/v3/projects/4bc201b9ec6c4e709a93c22574f29e79/users/f25a75d1055e4f10bee83e35645de45c/roles/a432dd4735944ecfab2fa5ce1705d356` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:43.013070 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6754a96-782e-42ae-8edd-4bd2478d66ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bc201b9ec6c4e709a93c22574f29e79, user_id=f25a75d1055e4f10bee83e35645de45c, role_id=a432dd4735944ecfab2fa5ce1705d356)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:43.015424 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6754a96-782e-42ae-8edd-4bd2478d66ab admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:43.027764 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1309/2651] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:32:42 2022] PUT /identity/v3/projects/4bc201b9ec6c4e709a93c22574f29e79/users/f25a75d1055e4f10bee83e35645de45c/roles/a432dd4735944ecfab2fa5ce1705d356 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:32:43.038279 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7888c2c4-01f5-42e5-b0d2-8af216bfa1e5 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:43.038572 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7888c2c4-01f5-42e5-b0d2-8af216bfa1e5 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:43.038852 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7888c2c4-01f5-42e5-b0d2-8af216bfa1e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:43.068774 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-7888c2c4-01f5-42e5-b0d2-8af216bfa1e5 None None] MFA Rules not processed for user `f25a75d1055e4f10bee83e35645de45c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:32:43.070179 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-7888c2c4-01f5-42e5-b0d2-8af216bfa1e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:43.156546 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1343/2652] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:32:43 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:32:43.236263 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6754a96-782e-42ae-8edd-4bd2478d66ab admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:43.238609 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-f6754a96-782e-42ae-8edd-4bd2478d66ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:43.300979 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6754a96-782e-42ae-8edd-4bd2478d66ab admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:43.304626 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b6ac37-cd83-464c-b708-e4b31213ae7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e31f2728056b460fa474478eb85ca1d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:43.305250 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b6ac37-cd83-464c-b708-e4b31213ae7e service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:43.305532 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b6ac37-cd83-464c-b708-e4b31213ae7e service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:43.305804 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b6ac37-cd83-464c-b708-e4b31213ae7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:43.373684 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b6ac37-cd83-464c-b708-e4b31213ae7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:43.375151 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b6ac37-cd83-464c-b708-e4b31213ae7e service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:43.389581 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1310/2653] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:32:43 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:32:53.153509 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=79 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:53.382908 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=60,keystone:INSERT=2 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:59.805015 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7888c2c4-01f5-42e5-b0d2-8af216bfa1e5 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:59.811289 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7888c2c4-01f5-42e5-b0d2-8af216bfa1e5 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:59.815565 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93391595-4869-4277-8139-ff577b4ad1e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e31f2728056b460fa474478eb85ca1d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:59.816209 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93391595-4869-4277-8139-ff577b4ad1e7 service cinder] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:59.816495 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93391595-4869-4277-8139-ff577b4ad1e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:59.816766 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93391595-4869-4277-8139-ff577b4ad1e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:59.827050 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93391595-4869-4277-8139-ff577b4ad1e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:59.828553 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93391595-4869-4277-8139-ff577b4ad1e7 service cinder] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:59.839388 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1344/2654] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:32:59 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:33:00.760475 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f89e55-f609-45a8-891a-fa92f77fe441 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:00.761147 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f89e55-f609-45a8-891a-fa92f77fe441 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:00.761329 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f89e55-f609-45a8-891a-fa92f77fe441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:00.784954 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-f0f89e55-f609-45a8-891a-fa92f77fe441 None None] MFA Rules not processed for user `b665d491470f4a0b95c13f51e19d2046`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:33:00.787382 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-f0f89e55-f609-45a8-891a-fa92f77fe441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:00.870138 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1311/2655] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:33:00 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:33:01.235911 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93391595-4869-4277-8139-ff577b4ad1e7 service cinder] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:01.237458 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-93391595-4869-4277-8139-ff577b4ad1e7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:01.297422 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93391595-4869-4277-8139-ff577b4ad1e7 service cinder] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:01.301757 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935ac935-24b0-49da-8af9-8c294f745cb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:01.302377 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935ac935-24b0-49da-8af9-8c294f745cb3 service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:01.302662 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935ac935-24b0-49da-8af9-8c294f745cb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:01.302944 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935ac935-24b0-49da-8af9-8c294f745cb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:01.305503 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-935ac935-24b0-49da-8af9-8c294f745cb3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:01.317658 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935ac935-24b0-49da-8af9-8c294f745cb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:01.319582 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935ac935-24b0-49da-8af9-8c294f745cb3 service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:01.328939 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1345/2656] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:33:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:33:10.525186 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f89e55-f609-45a8-891a-fa92f77fe441 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:10.526727 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-f0f89e55-f609-45a8-891a-fa92f77fe441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:10.595973 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f89e55-f609-45a8-891a-fa92f77fe441 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:10.601500 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f964d49-e8cb-4b6f-9857-2b56a5066d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:10.602457 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f964d49-e8cb-4b6f-9857-2b56a5066d41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:10.602731 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f964d49-e8cb-4b6f-9857-2b56a5066d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:10.603004 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f964d49-e8cb-4b6f-9857-2b56a5066d41 admin admin] PATH_INFO: `/v3/users/08e724743bb94e34a1c8a786097499cb` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:10.604257 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f964d49-e8cb-4b6f-9857-2b56a5066d41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08e724743bb94e34a1c8a786097499cb)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:10.606200 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f964d49-e8cb-4b6f-9857-2b56a5066d41 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:10.812039 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-3f964d49-e8cb-4b6f-9857-2b56a5066d41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08e724743bb94e34a1c8a786097499cb'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:10.825385 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-3f964d49-e8cb-4b6f-9857-2b56a5066d41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08e724743bb94e34a1c8a786097499cb'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:10.826113 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-3f964d49-e8cb-4b6f-9857-2b56a5066d41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08e724743bb94e34a1c8a786097499cb'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:10.834453 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-3f964d49-e8cb-4b6f-9857-2b56a5066d41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08e724743bb94e34a1c8a786097499cb'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:10.835136 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-3f964d49-e8cb-4b6f-9857-2b56a5066d41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08e724743bb94e34a1c8a786097499cb'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:10.889053 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1312/2657] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:33:10 2022] DELETE /identity/v3/users/08e724743bb94e34a1c8a786097499cb => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:33:10.893556 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bf484b-7940-4929-bfad-155f4bda4170 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:10.894118 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bf484b-7940-4929-bfad-155f4bda4170 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:10.894118 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bf484b-7940-4929-bfad-155f4bda4170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:10.913264 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-f2bf484b-7940-4929-bfad-155f4bda4170 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:33:10.914471 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-f2bf484b-7940-4929-bfad-155f4bda4170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:10.990348 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1346/2658] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:33:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:33:11.000738 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f964d49-e8cb-4b6f-9857-2b56a5066d41 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:11.002293 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-3f964d49-e8cb-4b6f-9857-2b56a5066d41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:11.048484 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:11.095435 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f964d49-e8cb-4b6f-9857-2b56a5066d41 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:11.101922 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45390d8e-da18-4b72-921f-7ddc70ae4986 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:11.102836 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45390d8e-da18-4b72-921f-7ddc70ae4986 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:11.103259 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45390d8e-da18-4b72-921f-7ddc70ae4986 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:11.103723 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45390d8e-da18-4b72-921f-7ddc70ae4986 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:11.118926 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45390d8e-da18-4b72-921f-7ddc70ae4986 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:11.121358 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45390d8e-da18-4b72-921f-7ddc70ae4986 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:11.137283 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1313/2659] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:33:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:33:11.616166 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bf484b-7940-4929-bfad-155f4bda4170 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:11.617972 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-f2bf484b-7940-4929-bfad-155f4bda4170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:11.637728 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bf484b-7940-4929-bfad-155f4bda4170 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:11.645183 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8bd07a-9235-46c4-8741-e2b9137af48a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:11.646225 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8bd07a-9235-46c4-8741-e2b9137af48a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:11.646539 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8bd07a-9235-46c4-8741-e2b9137af48a admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:11.646857 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8bd07a-9235-46c4-8741-e2b9137af48a admin admin] PATH_INFO: `/v3/projects/66698488e28e43a6b3b3f78abe6cec16` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:11.653425 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8bd07a-9235-46c4-8741-e2b9137af48a admin admin] RBAC: Authorizing `identity:delete_project(project_id=66698488e28e43a6b3b3f78abe6cec16)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:11.653425 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8bd07a-9235-46c4-8741-e2b9137af48a admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:11.770598 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-bb8bd07a-9235-46c4-8741-e2b9137af48a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66698488e28e43a6b3b3f78abe6cec16', 'request_id': 'req-bb8bd07a-9235-46c4-8741-e2b9137af48a'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:11.777286 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-bb8bd07a-9235-46c4-8741-e2b9137af48a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66698488e28e43a6b3b3f78abe6cec16', 'request_id': 'req-bb8bd07a-9235-46c4-8741-e2b9137af48a'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:11.856165 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-bb8bd07a-9235-46c4-8741-e2b9137af48a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66698488e28e43a6b3b3f78abe6cec16', 'request_id': 'req-bb8bd07a-9235-46c4-8741-e2b9137af48a'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:11.868627 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-bb8bd07a-9235-46c4-8741-e2b9137af48a admin admin] The token cache is being invalidate because project 66698488e28e43a6b3b3f78abe6cec16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:33:11.868832 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-bb8bd07a-9235-46c4-8741-e2b9137af48a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:11.869460 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-bb8bd07a-9235-46c4-8741-e2b9137af48a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:11.871029 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1347/2660] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:33:11 2022] DELETE /identity/v3/projects/66698488e28e43a6b3b3f78abe6cec16 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:33:11.876215 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd1f075-2cc1-40a5-93e4-83d2963405ab None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:11.876705 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd1f075-2cc1-40a5-93e4-83d2963405ab None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:11.876832 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd1f075-2cc1-40a5-93e4-83d2963405ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:11.899370 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-dbd1f075-2cc1-40a5-93e4-83d2963405ab None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:33:11.900591 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-dbd1f075-2cc1-40a5-93e4-83d2963405ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:11.963870 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1314/2661] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:33:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:33:11.973442 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dcf886-fbad-4dee-a5eb-0e01d29b5336 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:11.974025 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dcf886-fbad-4dee-a5eb-0e01d29b5336 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:11.974025 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dcf886-fbad-4dee-a5eb-0e01d29b5336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:11.994763 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-95dcf886-fbad-4dee-a5eb-0e01d29b5336 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:33:11.996004 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-95dcf886-fbad-4dee-a5eb-0e01d29b5336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:12.011744 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1348/2662] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:33:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:33:12.018610 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd1f075-2cc1-40a5-93e4-83d2963405ab None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:12.033019 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd1f075-2cc1-40a5-93e4-83d2963405ab None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:12.040385 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1037d1c1-ea59-4ecb-ac17-a5f8bb317e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:12.041499 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1037d1c1-ea59-4ecb-ac17-a5f8bb317e8b admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:12.041962 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1037d1c1-ea59-4ecb-ac17-a5f8bb317e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:12.042411 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1037d1c1-ea59-4ecb-ac17-a5f8bb317e8b admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:12.043667 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1037d1c1-ea59-4ecb-ac17-a5f8bb317e8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:33:12.044229 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1037d1c1-ea59-4ecb-ac17-a5f8bb317e8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:12.047205 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1037d1c1-ea59-4ecb-ac17-a5f8bb317e8b admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:12.070576 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1315/2663] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Aug 4 21:33:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:33:12.075264 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dcf886-fbad-4dee-a5eb-0e01d29b5336 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:12.085635 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dcf886-fbad-4dee-a5eb-0e01d29b5336 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:12.090755 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3491e9a8-2d18-40f9-8ae4-97e2fe5b7851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:12.091430 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3491e9a8-2d18-40f9-8ae4-97e2fe5b7851 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:12.091716 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3491e9a8-2d18-40f9-8ae4-97e2fe5b7851 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:12.091990 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3491e9a8-2d18-40f9-8ae4-97e2fe5b7851 admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:12.092659 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3491e9a8-2d18-40f9-8ae4-97e2fe5b7851 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:12.094718 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3491e9a8-2d18-40f9-8ae4-97e2fe5b7851 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:12.101591 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:12.205333 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1349/2664] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:33:12 2022] POST /identity/v3/projects => generated 367 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:33:12.210705 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1037d1c1-ea59-4ecb-ac17-a5f8bb317e8b admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:12.222623 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1037d1c1-ea59-4ecb-ac17-a5f8bb317e8b admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:12.292038 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6916895-621a-41b7-bd1c-511c1b958b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:12.292660 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6916895-621a-41b7-bd1c-511c1b958b26 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:12.292940 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6916895-621a-41b7-bd1c-511c1b958b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:12.293215 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6916895-621a-41b7-bd1c-511c1b958b26 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:12.293876 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6916895-621a-41b7-bd1c-511c1b958b26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:12.295853 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6916895-621a-41b7-bd1c-511c1b958b26 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:12.415385 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1316/2665] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:33:12 2022] POST /identity/v3/users => generated 284 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:33:12.425433 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3491e9a8-2d18-40f9-8ae4-97e2fe5b7851 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:12.436456 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3491e9a8-2d18-40f9-8ae4-97e2fe5b7851 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:12.441178 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f4639e-cf68-488b-830c-be92b6518ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:12.441800 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f4639e-cf68-488b-830c-be92b6518ea2 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:12.442082 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f4639e-cf68-488b-830c-be92b6518ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:12.442350 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f4639e-cf68-488b-830c-be92b6518ea2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:12.443076 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f4639e-cf68-488b-830c-be92b6518ea2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:12.444714 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f4639e-cf68-488b-830c-be92b6518ea2 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:12.455212 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1350/2666] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:33:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:33:12.460565 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6916895-621a-41b7-bd1c-511c1b958b26 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:12.474809 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6916895-621a-41b7-bd1c-511c1b958b26 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:12.480161 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bdc140-0259-4c6b-8aea-c44eeb67582b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:12.480866 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bdc140-0259-4c6b-8aea-c44eeb67582b admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:12.481147 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bdc140-0259-4c6b-8aea-c44eeb67582b admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:12.481418 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bdc140-0259-4c6b-8aea-c44eeb67582b admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:12.482171 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bdc140-0259-4c6b-8aea-c44eeb67582b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:12.484215 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bdc140-0259-4c6b-8aea-c44eeb67582b admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:12.501615 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1317/2667] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:33:12 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:33:12.507134 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f4639e-cf68-488b-830c-be92b6518ea2 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:12.518228 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f4639e-cf68-488b-830c-be92b6518ea2 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:12.525876 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6966c7d1-1c0a-4539-b2ea-7f7a4e4ee5dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:12.525876 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6966c7d1-1c0a-4539-b2ea-7f7a4e4ee5dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:12.525876 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6966c7d1-1c0a-4539-b2ea-7f7a4e4ee5dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:12.526445 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6966c7d1-1c0a-4539-b2ea-7f7a4e4ee5dc admin admin] PATH_INFO: `/v3/projects/b43abc1ee5a543ccb8930f235403a47e/users/c0645e94946f4d44b03b84a4665d2ac4/roles/8feebc60363c4feb870718b1721e5324` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:12.547267 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6966c7d1-1c0a-4539-b2ea-7f7a4e4ee5dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=b43abc1ee5a543ccb8930f235403a47e, user_id=c0645e94946f4d44b03b84a4665d2ac4, role_id=8feebc60363c4feb870718b1721e5324)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:12.550434 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6966c7d1-1c0a-4539-b2ea-7f7a4e4ee5dc admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:12.645194 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1351/2668] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:33:12 2022] PUT /identity/v3/projects/b43abc1ee5a543ccb8930f235403a47e/users/c0645e94946f4d44b03b84a4665d2ac4/roles/8feebc60363c4feb870718b1721e5324 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:33:12.656835 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18fe3df-c92a-4634-89b1-6fbb61f5549f None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:12.657244 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18fe3df-c92a-4634-89b1-6fbb61f5549f None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:12.657728 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18fe3df-c92a-4634-89b1-6fbb61f5549f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:12.692141 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-b18fe3df-c92a-4634-89b1-6fbb61f5549f None None] MFA Rules not processed for user `c0645e94946f4d44b03b84a4665d2ac4`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:33:12.694099 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-b18fe3df-c92a-4634-89b1-6fbb61f5549f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:12.716792 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:12.769666 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1318/2669] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:33:12 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:33:12.783649 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6966c7d1-1c0a-4539-b2ea-7f7a4e4ee5dc admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:12.786206 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-6966c7d1-1c0a-4539-b2ea-7f7a4e4ee5dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:12.851863 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6966c7d1-1c0a-4539-b2ea-7f7a4e4ee5dc admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:12.856178 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c188c8-187b-4270-a09e-191fb990f1e9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6885c7d929f4f8e96d3890724c3b53f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:12.857152 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c188c8-187b-4270-a09e-191fb990f1e9 service glance] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:12.857574 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c188c8-187b-4270-a09e-191fb990f1e9 service glance] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:12.857852 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c188c8-187b-4270-a09e-191fb990f1e9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:12.867932 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c188c8-187b-4270-a09e-191fb990f1e9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:12.870530 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c188c8-187b-4270-a09e-191fb990f1e9 service glance] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:12.881322 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1352/2670] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:33:12 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:33:14.468358 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18fe3df-c92a-4634-89b1-6fbb61f5549f None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:14.469961 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-b18fe3df-c92a-4634-89b1-6fbb61f5549f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:14.558766 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18fe3df-c92a-4634-89b1-6fbb61f5549f None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:14.566137 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522fd2bf-3dff-4012-a919-ae860edf614e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:14.566859 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522fd2bf-3dff-4012-a919-ae860edf614e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:14.567270 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522fd2bf-3dff-4012-a919-ae860edf614e admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:14.567703 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522fd2bf-3dff-4012-a919-ae860edf614e admin admin] PATH_INFO: `/v3/users/0f97b8f68c254c6bb60be280a7ec4993` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:14.569092 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522fd2bf-3dff-4012-a919-ae860edf614e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f97b8f68c254c6bb60be280a7ec4993)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:14.571115 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522fd2bf-3dff-4012-a919-ae860edf614e admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:14.857329 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-522fd2bf-3dff-4012-a919-ae860edf614e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f97b8f68c254c6bb60be280a7ec4993'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:14.884491 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-522fd2bf-3dff-4012-a919-ae860edf614e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f97b8f68c254c6bb60be280a7ec4993'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:14.890384 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-522fd2bf-3dff-4012-a919-ae860edf614e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f97b8f68c254c6bb60be280a7ec4993'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:14.897423 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-522fd2bf-3dff-4012-a919-ae860edf614e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f97b8f68c254c6bb60be280a7ec4993'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:14.898434 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-522fd2bf-3dff-4012-a919-ae860edf614e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f97b8f68c254c6bb60be280a7ec4993'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:14.982730 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1319/2671] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:33:14 2022] DELETE /identity/v3/users/0f97b8f68c254c6bb60be280a7ec4993 => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:33:14.990784 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2c9f5a-f7f7-4e9a-86ba-c85fdb373e8e None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:14.992604 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2c9f5a-f7f7-4e9a-86ba-c85fdb373e8e None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:14.994466 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2c9f5a-f7f7-4e9a-86ba-c85fdb373e8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:15.020082 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-aa2c9f5a-f7f7-4e9a-86ba-c85fdb373e8e None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:33:15.021748 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-aa2c9f5a-f7f7-4e9a-86ba-c85fdb373e8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:15.128368 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1353/2672] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:33:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:33:15.142415 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522fd2bf-3dff-4012-a919-ae860edf614e admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:15.144685 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-522fd2bf-3dff-4012-a919-ae860edf614e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:15.202021 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:15.214768 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522fd2bf-3dff-4012-a919-ae860edf614e admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:15.224596 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea6f944-27ce-4379-a65a-9086202169c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:15.225668 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea6f944-27ce-4379-a65a-9086202169c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:15.227609 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea6f944-27ce-4379-a65a-9086202169c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:15.227609 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea6f944-27ce-4379-a65a-9086202169c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:15.243059 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea6f944-27ce-4379-a65a-9086202169c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:15.244705 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea6f944-27ce-4379-a65a-9086202169c6 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:15.263090 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1320/2673] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:33:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:33:15.739348 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2c9f5a-f7f7-4e9a-86ba-c85fdb373e8e None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:15.739348 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-aa2c9f5a-f7f7-4e9a-86ba-c85fdb373e8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:15.751393 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2c9f5a-f7f7-4e9a-86ba-c85fdb373e8e None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:15.758074 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2267180f-ea94-438c-be4e-7dab49b43785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:15.758732 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2267180f-ea94-438c-be4e-7dab49b43785 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:15.759007 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2267180f-ea94-438c-be4e-7dab49b43785 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:15.759276 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2267180f-ea94-438c-be4e-7dab49b43785 admin admin] PATH_INFO: `/v3/projects/07d51f83113a4885a4162bf1f27a2633` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:15.760521 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2267180f-ea94-438c-be4e-7dab49b43785 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07d51f83113a4885a4162bf1f27a2633)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:15.762521 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2267180f-ea94-438c-be4e-7dab49b43785 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:15.881325 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:15.882919 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-2267180f-ea94-438c-be4e-7dab49b43785 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07d51f83113a4885a4162bf1f27a2633', 'request_id': 'req-2267180f-ea94-438c-be4e-7dab49b43785'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:15.892114 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-2267180f-ea94-438c-be4e-7dab49b43785 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07d51f83113a4885a4162bf1f27a2633', 'request_id': 'req-2267180f-ea94-438c-be4e-7dab49b43785'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:15.973351 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-2267180f-ea94-438c-be4e-7dab49b43785 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07d51f83113a4885a4162bf1f27a2633', 'request_id': 'req-2267180f-ea94-438c-be4e-7dab49b43785'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:15.985710 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-2267180f-ea94-438c-be4e-7dab49b43785 admin admin] The token cache is being invalidate because project 07d51f83113a4885a4162bf1f27a2633 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:33:15.986003 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-2267180f-ea94-438c-be4e-7dab49b43785 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:15.986652 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-2267180f-ea94-438c-be4e-7dab49b43785 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:15.988875 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1354/2674] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:33:15 2022] DELETE /identity/v3/projects/07d51f83113a4885a4162bf1f27a2633 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:33:15.993417 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea6f944-27ce-4379-a65a-9086202169c6 service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:16.005444 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea6f944-27ce-4379-a65a-9086202169c6 service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:16.077399 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee256d0-77ef-4680-9d44-13d5e05c5148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:16.078047 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee256d0-77ef-4680-9d44-13d5e05c5148 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:16.078326 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee256d0-77ef-4680-9d44-13d5e05c5148 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:16.078597 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee256d0-77ef-4680-9d44-13d5e05c5148 admin admin] PATH_INFO: `/v3/users/f25a75d1055e4f10bee83e35645de45c` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:16.079817 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee256d0-77ef-4680-9d44-13d5e05c5148 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f25a75d1055e4f10bee83e35645de45c)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:16.081817 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee256d0-77ef-4680-9d44-13d5e05c5148 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:16.282665 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-8ee256d0-77ef-4680-9d44-13d5e05c5148 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f25a75d1055e4f10bee83e35645de45c'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:16.303857 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-8ee256d0-77ef-4680-9d44-13d5e05c5148 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f25a75d1055e4f10bee83e35645de45c'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:16.305192 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-8ee256d0-77ef-4680-9d44-13d5e05c5148 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f25a75d1055e4f10bee83e35645de45c'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:16.314845 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-8ee256d0-77ef-4680-9d44-13d5e05c5148 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f25a75d1055e4f10bee83e35645de45c'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:16.315753 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-8ee256d0-77ef-4680-9d44-13d5e05c5148 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f25a75d1055e4f10bee83e35645de45c'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:16.424392 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1321/2675] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:33:15 2022] DELETE /identity/v3/users/f25a75d1055e4f10bee83e35645de45c => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:33:16.902826 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2267180f-ea94-438c-be4e-7dab49b43785 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:16.904359 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-2267180f-ea94-438c-be4e-7dab49b43785 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:16.931429 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2267180f-ea94-438c-be4e-7dab49b43785 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:17.015847 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e932191-f794-43fc-855b-48e8effbc88f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:17.016855 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e932191-f794-43fc-855b-48e8effbc88f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:17.017327 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e932191-f794-43fc-855b-48e8effbc88f admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:17.017714 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e932191-f794-43fc-855b-48e8effbc88f admin admin] PATH_INFO: `/v3/projects/4bc201b9ec6c4e709a93c22574f29e79` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:17.022348 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e932191-f794-43fc-855b-48e8effbc88f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bc201b9ec6c4e709a93c22574f29e79)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:17.025039 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e932191-f794-43fc-855b-48e8effbc88f admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:17.186671 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-4e932191-f794-43fc-855b-48e8effbc88f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bc201b9ec6c4e709a93c22574f29e79', 'request_id': 'req-4e932191-f794-43fc-855b-48e8effbc88f'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:17.194624 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-4e932191-f794-43fc-855b-48e8effbc88f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bc201b9ec6c4e709a93c22574f29e79', 'request_id': 'req-4e932191-f794-43fc-855b-48e8effbc88f'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:17.283419 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-4e932191-f794-43fc-855b-48e8effbc88f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bc201b9ec6c4e709a93c22574f29e79', 'request_id': 'req-4e932191-f794-43fc-855b-48e8effbc88f'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:17.298015 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-4e932191-f794-43fc-855b-48e8effbc88f admin admin] The token cache is being invalidate because project 4bc201b9ec6c4e709a93c22574f29e79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:33:17.298443 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-4e932191-f794-43fc-855b-48e8effbc88f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:17.299178 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-4e932191-f794-43fc-855b-48e8effbc88f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:17.301708 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1355/2676] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:33:16 2022] DELETE /identity/v3/projects/4bc201b9ec6c4e709a93c22574f29e79 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:33:17.308943 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcce706-d015-4205-a736-dbce3e17979b None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:17.309685 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcce706-d015-4205-a736-dbce3e17979b None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:17.309685 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcce706-d015-4205-a736-dbce3e17979b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:17.345600 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-0dcce706-d015-4205-a736-dbce3e17979b None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:33:17.347219 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-0dcce706-d015-4205-a736-dbce3e17979b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:17.367707 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e932191-f794-43fc-855b-48e8effbc88f admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:17.369681 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-4e932191-f794-43fc-855b-48e8effbc88f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:17.483471 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1322/2677] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:33:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:33:17.493044 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3951011-3d30-4e4e-8fe6-173599b2657f None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:17.493537 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3951011-3d30-4e4e-8fe6-173599b2657f None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:17.493942 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3951011-3d30-4e4e-8fe6-173599b2657f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:17.508392 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:17.516167 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:17.524268 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-e3951011-3d30-4e4e-8fe6-173599b2657f None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:33:17.525612 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-e3951011-3d30-4e4e-8fe6-173599b2657f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:17.535395 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1323/2678] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:33:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:33:17.541095 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3951011-3d30-4e4e-8fe6-173599b2657f None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:17.558393 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3951011-3d30-4e4e-8fe6-173599b2657f None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:17.561785 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e932191-f794-43fc-855b-48e8effbc88f admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:17.566725 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4c6213-0319-47c7-8ac7-7676b6428954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:17.568232 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c6213-0319-47c7-8ac7-7676b6428954 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:17.568993 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c6213-0319-47c7-8ac7-7676b6428954 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:17.569242 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92da60a5-6b96-4dd2-92c4-27462c0dc2bb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67d44a0098ab4c36b14b36e9e59ae75c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:17.569883 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92da60a5-6b96-4dd2-92c4-27462c0dc2bb service placement] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:17.570171 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92da60a5-6b96-4dd2-92c4-27462c0dc2bb service placement] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:17.570452 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92da60a5-6b96-4dd2-92c4-27462c0dc2bb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:17.571008 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c6213-0319-47c7-8ac7-7676b6428954 admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:17.572831 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-92da60a5-6b96-4dd2-92c4-27462c0dc2bb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:17.573759 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4c6213-0319-47c7-8ac7-7676b6428954 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:33:17.574520 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4c6213-0319-47c7-8ac7-7676b6428954 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:17.578084 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4c6213-0319-47c7-8ac7-7676b6428954 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:17.589394 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92da60a5-6b96-4dd2-92c4-27462c0dc2bb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:17.591275 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92da60a5-6b96-4dd2-92c4-27462c0dc2bb service placement] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:17.600641 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1324/2679] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Aug 4 21:33:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:33:17.604852 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4c6213-0319-47c7-8ac7-7676b6428954 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:17.612013 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1356/2680] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:33:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:33:17.616512 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4c6213-0319-47c7-8ac7-7676b6428954 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:17.623056 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c7e094-73cb-47c0-8b6d-e21a8c2d8fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:17.623737 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c7e094-73cb-47c0-8b6d-e21a8c2d8fa5 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:17.624085 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c7e094-73cb-47c0-8b6d-e21a8c2d8fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:17.624363 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c7e094-73cb-47c0-8b6d-e21a8c2d8fa5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:17.625049 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c7e094-73cb-47c0-8b6d-e21a8c2d8fa5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:17.626998 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c7e094-73cb-47c0-8b6d-e21a8c2d8fa5 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:17.739256 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1325/2681] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:33:17 2022] POST /identity/v3/projects => generated 385 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:33:17.743479 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92da60a5-6b96-4dd2-92c4-27462c0dc2bb service placement] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:17.754594 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92da60a5-6b96-4dd2-92c4-27462c0dc2bb service placement] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:17.825230 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c1904f-6641-4df6-8716-04035f974a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:17.825908 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c1904f-6641-4df6-8716-04035f974a29 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:17.826298 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c1904f-6641-4df6-8716-04035f974a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:17.826578 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c1904f-6641-4df6-8716-04035f974a29 admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:17.827268 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c1904f-6641-4df6-8716-04035f974a29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:17.829243 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c1904f-6641-4df6-8716-04035f974a29 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:17.953622 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1357/2682] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:33:17 2022] POST /identity/v3/users => generated 293 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:33:17.958638 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c7e094-73cb-47c0-8b6d-e21a8c2d8fa5 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:17.970524 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c7e094-73cb-47c0-8b6d-e21a8c2d8fa5 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:17.976706 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8480285d-4ccd-4e8e-a01d-3a02bf467e10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:17.977383 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8480285d-4ccd-4e8e-a01d-3a02bf467e10 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:17.977664 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8480285d-4ccd-4e8e-a01d-3a02bf467e10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:17.977970 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8480285d-4ccd-4e8e-a01d-3a02bf467e10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:17.978702 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8480285d-4ccd-4e8e-a01d-3a02bf467e10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:17.980365 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8480285d-4ccd-4e8e-a01d-3a02bf467e10 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:17.991717 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1326/2683] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:33:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:33:17.996291 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c1904f-6641-4df6-8716-04035f974a29 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:18.008414 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c1904f-6641-4df6-8716-04035f974a29 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:18.013823 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9b6a35-7e72-4c4e-91a8-2117909b25d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:18.014496 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9b6a35-7e72-4c4e-91a8-2117909b25d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:18.014782 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9b6a35-7e72-4c4e-91a8-2117909b25d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:18.015056 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9b6a35-7e72-4c4e-91a8-2117909b25d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:18.015800 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9b6a35-7e72-4c4e-91a8-2117909b25d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:18.017472 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9b6a35-7e72-4c4e-91a8-2117909b25d6 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:18.029402 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1358/2684] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:33:17 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:33:18.033591 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8480285d-4ccd-4e8e-a01d-3a02bf467e10 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:18.044514 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8480285d-4ccd-4e8e-a01d-3a02bf467e10 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:18.049343 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6829cc9-7e61-4699-9e5c-ed389dcb5891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:18.050754 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6829cc9-7e61-4699-9e5c-ed389dcb5891 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:18.051098 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6829cc9-7e61-4699-9e5c-ed389dcb5891 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:18.051426 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6829cc9-7e61-4699-9e5c-ed389dcb5891 admin admin] PATH_INFO: `/v3/projects/0729adf9197b40a5b98e805510fa5a75/users/70977370ac8c43d4a9daa05036fd6074/roles/8feebc60363c4feb870718b1721e5324` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:18.068995 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6829cc9-7e61-4699-9e5c-ed389dcb5891 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0729adf9197b40a5b98e805510fa5a75, user_id=70977370ac8c43d4a9daa05036fd6074, role_id=8feebc60363c4feb870718b1721e5324)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:18.072600 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6829cc9-7e61-4699-9e5c-ed389dcb5891 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:18.147717 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1327/2685] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:33:18 2022] PUT /identity/v3/projects/0729adf9197b40a5b98e805510fa5a75/users/70977370ac8c43d4a9daa05036fd6074/roles/8feebc60363c4feb870718b1721e5324 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:33:18.158817 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c65e90-adcb-44c2-8a92-145fb9989342 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:18.159950 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c65e90-adcb-44c2-8a92-145fb9989342 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:18.160361 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c65e90-adcb-44c2-8a92-145fb9989342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:18.190474 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-35c65e90-adcb-44c2-8a92-145fb9989342 None None] MFA Rules not processed for user `70977370ac8c43d4a9daa05036fd6074`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:33:18.192731 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-35c65e90-adcb-44c2-8a92-145fb9989342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:18.271990 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1359/2686] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:33:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:33:18.285649 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6829cc9-7e61-4699-9e5c-ed389dcb5891 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:18.286750 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-d6829cc9-7e61-4699-9e5c-ed389dcb5891 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:18.351849 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6829cc9-7e61-4699-9e5c-ed389dcb5891 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:18.355847 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ba177a-525d-4c48-9fb3-f07aa75720b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e31f2728056b460fa474478eb85ca1d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:18.356482 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ba177a-525d-4c48-9fb3-f07aa75720b4 service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:18.356769 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ba177a-525d-4c48-9fb3-f07aa75720b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:18.357046 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ba177a-525d-4c48-9fb3-f07aa75720b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:18.366698 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ba177a-525d-4c48-9fb3-f07aa75720b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:18.368187 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ba177a-525d-4c48-9fb3-f07aa75720b4 service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:18.378109 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1328/2687] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:33:18 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:33:18.398202 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c65e90-adcb-44c2-8a92-145fb9989342 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:18.447782 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ba177a-525d-4c48-9fb3-f07aa75720b4 service cinder] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:18.449466 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-49ba177a-525d-4c48-9fb3-f07aa75720b4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:18.456724 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:18.479007 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c65e90-adcb-44c2-8a92-145fb9989342 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:18.485812 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a262d8-7cff-40df-8212-0b5f877b7a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:18.486492 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a262d8-7cff-40df-8212-0b5f877b7a8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:18.486779 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a262d8-7cff-40df-8212-0b5f877b7a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:18.487104 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a262d8-7cff-40df-8212-0b5f877b7a8f admin admin] PATH_INFO: `/v3/users/70977370ac8c43d4a9daa05036fd6074` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:18.489796 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a262d8-7cff-40df-8212-0b5f877b7a8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=70977370ac8c43d4a9daa05036fd6074)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:18.492849 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a262d8-7cff-40df-8212-0b5f877b7a8f admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:18.539780 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ba177a-525d-4c48-9fb3-f07aa75720b4 service cinder] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:18.546006 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db956536-e7ba-4cbc-9c66-0ab67078dd85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:18.546531 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db956536-e7ba-4cbc-9c66-0ab67078dd85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:18.546907 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db956536-e7ba-4cbc-9c66-0ab67078dd85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:18.547183 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db956536-e7ba-4cbc-9c66-0ab67078dd85 admin admin] PATH_INFO: `/v3/users/dd60cb83752d48cb8275d7efd2060e9e` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:18.548628 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db956536-e7ba-4cbc-9c66-0ab67078dd85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd60cb83752d48cb8275d7efd2060e9e)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:18.551441 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db956536-e7ba-4cbc-9c66-0ab67078dd85 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:18.686551 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-57a262d8-7cff-40df-8212-0b5f877b7a8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70977370ac8c43d4a9daa05036fd6074'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:18.698411 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:18.707027 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-57a262d8-7cff-40df-8212-0b5f877b7a8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70977370ac8c43d4a9daa05036fd6074'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:18.707769 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-57a262d8-7cff-40df-8212-0b5f877b7a8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70977370ac8c43d4a9daa05036fd6074'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:18.708401 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-57a262d8-7cff-40df-8212-0b5f877b7a8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70977370ac8c43d4a9daa05036fd6074'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:18.715040 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-db956536-e7ba-4cbc-9c66-0ab67078dd85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd60cb83752d48cb8275d7efd2060e9e'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:18.726834 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-db956536-e7ba-4cbc-9c66-0ab67078dd85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd60cb83752d48cb8275d7efd2060e9e'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:18.727644 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-db956536-e7ba-4cbc-9c66-0ab67078dd85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd60cb83752d48cb8275d7efd2060e9e'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:18.734662 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-db956536-e7ba-4cbc-9c66-0ab67078dd85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd60cb83752d48cb8275d7efd2060e9e'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:18.735401 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-db956536-e7ba-4cbc-9c66-0ab67078dd85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd60cb83752d48cb8275d7efd2060e9e'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:18.809061 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-57a262d8-7cff-40df-8212-0b5f877b7a8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70977370ac8c43d4a9daa05036fd6074'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:18.832123 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1360/2688] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:33:18 2022] DELETE /identity/v3/users/70977370ac8c43d4a9daa05036fd6074 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:33:18.836781 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da651845-cac5-4c2f-85cc-4be11f078797 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:18.837072 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da651845-cac5-4c2f-85cc-4be11f078797 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:18.837351 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da651845-cac5-4c2f-85cc-4be11f078797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:18.857352 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-da651845-cac5-4c2f-85cc-4be11f078797 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:33:18.858613 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-da651845-cac5-4c2f-85cc-4be11f078797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:18.877393 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1329/2689] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:33:18 2022] DELETE /identity/v3/users/dd60cb83752d48cb8275d7efd2060e9e => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:33:18.884687 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d24613-1e6b-4690-92b7-b3def0c61e98 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:18.885818 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d24613-1e6b-4690-92b7-b3def0c61e98 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:18.886435 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d24613-1e6b-4690-92b7-b3def0c61e98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:18.928740 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-15d24613-1e6b-4690-92b7-b3def0c61e98 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:33:18.932275 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-15d24613-1e6b-4690-92b7-b3def0c61e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:18.983503 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1361/2690] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:33:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:33:19.001888 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da651845-cac5-4c2f-85cc-4be11f078797 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:19.005303 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-da651845-cac5-4c2f-85cc-4be11f078797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:19.055255 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1330/2691] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:33:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:33:19.067869 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d24613-1e6b-4690-92b7-b3def0c61e98 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:19.069364 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-15d24613-1e6b-4690-92b7-b3def0c61e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:19.095732 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da651845-cac5-4c2f-85cc-4be11f078797 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:19.115105 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:19.157464 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d24613-1e6b-4690-92b7-b3def0c61e98 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:19.165172 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a04dc1-73ac-43a0-8c42-7d71c03cc4cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:19.166560 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a04dc1-73ac-43a0-8c42-7d71c03cc4cb service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:19.167172 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a04dc1-73ac-43a0-8c42-7d71c03cc4cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:19.167811 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a04dc1-73ac-43a0-8c42-7d71c03cc4cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:19.178020 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2816f8ca-2092-47e4-9fa5-4a33c56af399 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:19.178736 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2816f8ca-2092-47e4-9fa5-4a33c56af399 service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:19.179133 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2816f8ca-2092-47e4-9fa5-4a33c56af399 service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:19.179287 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2816f8ca-2092-47e4-9fa5-4a33c56af399 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:19.187363 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a04dc1-73ac-43a0-8c42-7d71c03cc4cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:19.187363 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a04dc1-73ac-43a0-8c42-7d71c03cc4cb service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:19.191785 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2816f8ca-2092-47e4-9fa5-4a33c56af399 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:19.193168 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2816f8ca-2092-47e4-9fa5-4a33c56af399 service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:19.206873 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1362/2692] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:33:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:33:19.209074 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1331/2693] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:33:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:33:19.786986 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2816f8ca-2092-47e4-9fa5-4a33c56af399 service neutron] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:19.788535 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-2816f8ca-2092-47e4-9fa5-4a33c56af399 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:19.807199 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2816f8ca-2092-47e4-9fa5-4a33c56af399 service neutron] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:19.813529 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0403e03b-c9d1-4186-89ee-c00163377384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:19.813948 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0403e03b-c9d1-4186-89ee-c00163377384 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:19.814245 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0403e03b-c9d1-4186-89ee-c00163377384 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:19.814515 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0403e03b-c9d1-4186-89ee-c00163377384 admin admin] PATH_INFO: `/v3/projects/0729adf9197b40a5b98e805510fa5a75` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:19.816224 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0403e03b-c9d1-4186-89ee-c00163377384 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0729adf9197b40a5b98e805510fa5a75)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:19.818709 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0403e03b-c9d1-4186-89ee-c00163377384 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:19.851547 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a04dc1-73ac-43a0-8c42-7d71c03cc4cb service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:19.854050 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-81a04dc1-73ac-43a0-8c42-7d71c03cc4cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:19.877026 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a04dc1-73ac-43a0-8c42-7d71c03cc4cb service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:19.882089 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6796f9-2740-4872-8daa-6de23997c793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:19.882745 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6796f9-2740-4872-8daa-6de23997c793 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:19.883021 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6796f9-2740-4872-8daa-6de23997c793 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:19.883291 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6796f9-2740-4872-8daa-6de23997c793 admin admin] PATH_INFO: `/v3/projects/4b753dad8a354b8895937eb7c40b935f` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:19.884532 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6796f9-2740-4872-8daa-6de23997c793 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b753dad8a354b8895937eb7c40b935f)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:19.886464 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6796f9-2740-4872-8daa-6de23997c793 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:19.978204 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-0403e03b-c9d1-4186-89ee-c00163377384 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0729adf9197b40a5b98e805510fa5a75', 'request_id': 'req-0403e03b-c9d1-4186-89ee-c00163377384'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:19.984457 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-0403e03b-c9d1-4186-89ee-c00163377384 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0729adf9197b40a5b98e805510fa5a75', 'request_id': 'req-0403e03b-c9d1-4186-89ee-c00163377384'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:20.058536 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-9d6796f9-2740-4872-8daa-6de23997c793 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b753dad8a354b8895937eb7c40b935f', 'request_id': 'req-9d6796f9-2740-4872-8daa-6de23997c793'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:20.060425 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:20.066797 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-9d6796f9-2740-4872-8daa-6de23997c793 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b753dad8a354b8895937eb7c40b935f', 'request_id': 'req-9d6796f9-2740-4872-8daa-6de23997c793'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:20.154229 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-0403e03b-c9d1-4186-89ee-c00163377384 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0729adf9197b40a5b98e805510fa5a75', 'request_id': 'req-0403e03b-c9d1-4186-89ee-c00163377384'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:20.168428 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-0403e03b-c9d1-4186-89ee-c00163377384 admin admin] The token cache is being invalidate because project 0729adf9197b40a5b98e805510fa5a75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:33:20.168701 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-0403e03b-c9d1-4186-89ee-c00163377384 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:20.169268 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-0403e03b-c9d1-4186-89ee-c00163377384 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:20.171132 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1363/2694] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:33:19 2022] DELETE /identity/v3/projects/0729adf9197b40a5b98e805510fa5a75 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:33:20.176805 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b109792-690b-4f82-9d78-20bfc5998bb0 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:20.177091 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b109792-690b-4f82-9d78-20bfc5998bb0 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:20.177367 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b109792-690b-4f82-9d78-20bfc5998bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:20.211377 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-4b109792-690b-4f82-9d78-20bfc5998bb0 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:33:20.214295 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-4b109792-690b-4f82-9d78-20bfc5998bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:20.260860 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-9d6796f9-2740-4872-8daa-6de23997c793 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b753dad8a354b8895937eb7c40b935f', 'request_id': 'req-9d6796f9-2740-4872-8daa-6de23997c793'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:20.288864 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-9d6796f9-2740-4872-8daa-6de23997c793 admin admin] The token cache is being invalidate because project 4b753dad8a354b8895937eb7c40b935f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:33:20.289295 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-9d6796f9-2740-4872-8daa-6de23997c793 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:20.289995 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-9d6796f9-2740-4872-8daa-6de23997c793 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:20.297509 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1332/2695] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:33:19 2022] DELETE /identity/v3/projects/4b753dad8a354b8895937eb7c40b935f => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:33:20.298540 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6796f9-2740-4872-8daa-6de23997c793 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:20.301351 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-9d6796f9-2740-4872-8daa-6de23997c793 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:20.352601 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1364/2696] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:33:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:33:20.364174 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c06760-899f-449e-81ef-5868a182d798 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:20.364536 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c06760-899f-449e-81ef-5868a182d798 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:20.364902 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c06760-899f-449e-81ef-5868a182d798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:20.398140 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-d1c06760-899f-449e-81ef-5868a182d798 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:33:20.402291 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-d1c06760-899f-449e-81ef-5868a182d798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:20.413463 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1365/2697] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:33:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:33:20.430417 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c06760-899f-449e-81ef-5868a182d798 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:20.430847 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6796f9-2740-4872-8daa-6de23997c793 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:20.432095 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4261c4-2139-47b3-ba21-257301a7aa25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:20.433449 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4261c4-2139-47b3-ba21-257301a7aa25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:20.433741 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4261c4-2139-47b3-ba21-257301a7aa25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:20.434017 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4261c4-2139-47b3-ba21-257301a7aa25 admin admin] PATH_INFO: `/v3/users/26fff62009e34be6a5da542e4237de7b` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:20.435880 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c06760-899f-449e-81ef-5868a182d798 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:20.436136 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4261c4-2139-47b3-ba21-257301a7aa25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26fff62009e34be6a5da542e4237de7b)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:20.439455 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4261c4-2139-47b3-ba21-257301a7aa25 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:20.447459 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff911be-00df-414e-bce7-86a339f058c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:20.448970 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff911be-00df-414e-bce7-86a339f058c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:20.448970 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff911be-00df-414e-bce7-86a339f058c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:20.448970 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff911be-00df-414e-bce7-86a339f058c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:20.449867 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff911be-00df-414e-bce7-86a339f058c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:33:20.450271 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff911be-00df-414e-bce7-86a339f058c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:20.452287 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff911be-00df-414e-bce7-86a339f058c9 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:20.475484 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1366/2698] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Aug 4 21:33:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:33:20.481995 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff911be-00df-414e-bce7-86a339f058c9 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:20.496110 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff911be-00df-414e-bce7-86a339f058c9 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:20.505896 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0878add-ad12-4feb-87c6-4fc23773ecbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:20.506651 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0878add-ad12-4feb-87c6-4fc23773ecbe admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:20.507081 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0878add-ad12-4feb-87c6-4fc23773ecbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:20.507547 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0878add-ad12-4feb-87c6-4fc23773ecbe admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:20.508616 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0878add-ad12-4feb-87c6-4fc23773ecbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:20.511938 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0878add-ad12-4feb-87c6-4fc23773ecbe admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:20.596500 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1367/2699] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:33:20 2022] POST /identity/v3/projects => generated 385 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:33:20.598862 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0878add-ad12-4feb-87c6-4fc23773ecbe admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:20.609535 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:20.613131 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-dc4261c4-2139-47b3-ba21-257301a7aa25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26fff62009e34be6a5da542e4237de7b'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:20.653489 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-dc4261c4-2139-47b3-ba21-257301a7aa25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26fff62009e34be6a5da542e4237de7b'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:20.655026 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-dc4261c4-2139-47b3-ba21-257301a7aa25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26fff62009e34be6a5da542e4237de7b'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:20.669639 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-dc4261c4-2139-47b3-ba21-257301a7aa25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26fff62009e34be6a5da542e4237de7b'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:20.670715 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-dc4261c4-2139-47b3-ba21-257301a7aa25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26fff62009e34be6a5da542e4237de7b'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:20.712560 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0878add-ad12-4feb-87c6-4fc23773ecbe admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:20.717419 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ead71d9-1291-4761-ac09-3b5710b46cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:20.718085 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ead71d9-1291-4761-ac09-3b5710b46cfa admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:20.718360 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ead71d9-1291-4761-ac09-3b5710b46cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:20.718635 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ead71d9-1291-4761-ac09-3b5710b46cfa admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:20.719307 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ead71d9-1291-4761-ac09-3b5710b46cfa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:20.721324 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ead71d9-1291-4761-ac09-3b5710b46cfa admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:20.773675 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1333/2700] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:33:20 2022] DELETE /identity/v3/users/26fff62009e34be6a5da542e4237de7b => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:33:20.845476 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1368/2701] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:33:20 2022] POST /identity/v3/users => generated 293 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:33:20.851460 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4261c4-2139-47b3-ba21-257301a7aa25 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:20.854278 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-dc4261c4-2139-47b3-ba21-257301a7aa25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:20.947741 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:20.954705 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4261c4-2139-47b3-ba21-257301a7aa25 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:20.963039 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9194ed6f-776e-4ac8-b348-24fa17c1b792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:20.963691 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9194ed6f-776e-4ac8-b348-24fa17c1b792 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:20.964033 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9194ed6f-776e-4ac8-b348-24fa17c1b792 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:20.964389 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9194ed6f-776e-4ac8-b348-24fa17c1b792 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:20.965609 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9194ed6f-776e-4ac8-b348-24fa17c1b792 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:20.967307 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9194ed6f-776e-4ac8-b348-24fa17c1b792 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:20.980199 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1334/2702] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:33:20 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:33:20.985125 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ead71d9-1291-4761-ac09-3b5710b46cfa admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:21.001595 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ead71d9-1291-4761-ac09-3b5710b46cfa admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:21.010641 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c261b03d-710d-4054-a890-bf6c7550a0ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:21.012383 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c261b03d-710d-4054-a890-bf6c7550a0ae admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:21.013075 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c261b03d-710d-4054-a890-bf6c7550a0ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:21.013649 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c261b03d-710d-4054-a890-bf6c7550a0ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:21.016831 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c261b03d-710d-4054-a890-bf6c7550a0ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:21.016831 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c261b03d-710d-4054-a890-bf6c7550a0ae admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:21.030015 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1369/2703] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:33:20 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:33:21.034506 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9194ed6f-776e-4ac8-b348-24fa17c1b792 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:21.057076 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9194ed6f-776e-4ac8-b348-24fa17c1b792 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:21.068363 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf2be5c-8080-42dc-bd44-601091d9faae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:21.070015 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf2be5c-8080-42dc-bd44-601091d9faae admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:21.070556 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf2be5c-8080-42dc-bd44-601091d9faae admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:21.071103 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf2be5c-8080-42dc-bd44-601091d9faae admin admin] PATH_INFO: `/v3/projects/9209deeb8fc84dffaf32be1451d82e7a/users/2aab5b8821ca41509a424d73cd322453/roles/8feebc60363c4feb870718b1721e5324` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:21.104604 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf2be5c-8080-42dc-bd44-601091d9faae admin admin] RBAC: Authorizing `identity:create_grant(project_id=9209deeb8fc84dffaf32be1451d82e7a, user_id=2aab5b8821ca41509a424d73cd322453, role_id=8feebc60363c4feb870718b1721e5324)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:21.109893 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf2be5c-8080-42dc-bd44-601091d9faae admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:21.220888 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1335/2704] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:33:21 2022] PUT /identity/v3/projects/9209deeb8fc84dffaf32be1451d82e7a/users/2aab5b8821ca41509a424d73cd322453/roles/8feebc60363c4feb870718b1721e5324 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:33:21.300722 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eedfba-8529-46f3-a186-037c2e90a5ab None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:21.301894 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eedfba-8529-46f3-a186-037c2e90a5ab None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:21.302564 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eedfba-8529-46f3-a186-037c2e90a5ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:21.344501 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-b1eedfba-8529-46f3-a186-037c2e90a5ab None None] MFA Rules not processed for user `2aab5b8821ca41509a424d73cd322453`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:33:21.347261 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-b1eedfba-8529-46f3-a186-037c2e90a5ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:21.354538 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf2be5c-8080-42dc-bd44-601091d9faae admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:21.356716 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-daf2be5c-8080-42dc-bd44-601091d9faae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:21.476059 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1370/2705] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:33:21 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:33:21.510826 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eedfba-8529-46f3-a186-037c2e90a5ab None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:21.512464 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-b1eedfba-8529-46f3-a186-037c2e90a5ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:21.513734 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf2be5c-8080-42dc-bd44-601091d9faae admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:21.521220 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f62264-fedd-48a9-87c1-f1c5d8e124e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:21.521985 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f62264-fedd-48a9-87c1-f1c5d8e124e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:21.522434 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f62264-fedd-48a9-87c1-f1c5d8e124e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:21.522884 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f62264-fedd-48a9-87c1-f1c5d8e124e1 admin admin] PATH_INFO: `/v3/projects/e6a2465c3cb54aaab40f5295e1f54ef2` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:21.524423 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f62264-fedd-48a9-87c1-f1c5d8e124e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6a2465c3cb54aaab40f5295e1f54ef2)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:21.526587 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f62264-fedd-48a9-87c1-f1c5d8e124e1 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:21.582192 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eedfba-8529-46f3-a186-037c2e90a5ab None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:21.586464 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35b84fe-0b56-4f21-8f0e-dcc97f171304 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e31f2728056b460fa474478eb85ca1d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:21.587098 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35b84fe-0b56-4f21-8f0e-dcc97f171304 service cinder] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:21.587423 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35b84fe-0b56-4f21-8f0e-dcc97f171304 service cinder] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:21.588131 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35b84fe-0b56-4f21-8f0e-dcc97f171304 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:21.597235 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35b84fe-0b56-4f21-8f0e-dcc97f171304 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:21.598675 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35b84fe-0b56-4f21-8f0e-dcc97f171304 service cinder] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:21.608484 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1371/2706] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:33:21 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:33:21.640424 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-d0f62264-fedd-48a9-87c1-f1c5d8e124e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6a2465c3cb54aaab40f5295e1f54ef2', 'request_id': 'req-d0f62264-fedd-48a9-87c1-f1c5d8e124e1'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:21.649473 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-d0f62264-fedd-48a9-87c1-f1c5d8e124e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6a2465c3cb54aaab40f5295e1f54ef2', 'request_id': 'req-d0f62264-fedd-48a9-87c1-f1c5d8e124e1'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:21.767141 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-d0f62264-fedd-48a9-87c1-f1c5d8e124e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6a2465c3cb54aaab40f5295e1f54ef2', 'request_id': 'req-d0f62264-fedd-48a9-87c1-f1c5d8e124e1'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:21.782678 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-d0f62264-fedd-48a9-87c1-f1c5d8e124e1 admin admin] The token cache is being invalidate because project e6a2465c3cb54aaab40f5295e1f54ef2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:33:21.782972 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-d0f62264-fedd-48a9-87c1-f1c5d8e124e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:21.783656 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-d0f62264-fedd-48a9-87c1-f1c5d8e124e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:21.785278 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1336/2707] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:33:21 2022] DELETE /identity/v3/projects/e6a2465c3cb54aaab40f5295e1f54ef2 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:33:21.790627 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972e8f1d-6944-4bd2-af94-330e2811ba41 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:21.790912 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972e8f1d-6944-4bd2-af94-330e2811ba41 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:21.791183 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972e8f1d-6944-4bd2-af94-330e2811ba41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:21.820337 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-972e8f1d-6944-4bd2-af94-330e2811ba41 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:33:21.822379 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-972e8f1d-6944-4bd2-af94-330e2811ba41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:21.835204 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:21.896649 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1372/2708] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:33:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:33:21.913824 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e193af3-489b-4f11-9452-91147262aacf None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:21.914562 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e193af3-489b-4f11-9452-91147262aacf None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:21.914562 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e193af3-489b-4f11-9452-91147262aacf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:21.937762 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-8e193af3-489b-4f11-9452-91147262aacf None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:33:21.938994 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-8e193af3-489b-4f11-9452-91147262aacf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:21.949496 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1337/2709] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:33:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:33:21.955332 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972e8f1d-6944-4bd2-af94-330e2811ba41 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:21.968576 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972e8f1d-6944-4bd2-af94-330e2811ba41 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:21.974918 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab53472-8d1b-4a8f-898f-0720ea9f2bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:21.976052 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab53472-8d1b-4a8f-898f-0720ea9f2bac admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:21.976518 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab53472-8d1b-4a8f-898f-0720ea9f2bac admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:21.976805 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab53472-8d1b-4a8f-898f-0720ea9f2bac admin admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:21.978027 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab53472-8d1b-4a8f-898f-0720ea9f2bac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:33:21.978628 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab53472-8d1b-4a8f-898f-0720ea9f2bac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:21.981748 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab53472-8d1b-4a8f-898f-0720ea9f2bac admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:22.006943 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1373/2710] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Aug 4 21:33:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:33:22.009828 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e193af3-489b-4f11-9452-91147262aacf None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:22.021246 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e193af3-489b-4f11-9452-91147262aacf None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:22.026632 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16c8a04-337e-4935-85e0-0fd247eb1dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:22.027358 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16c8a04-337e-4935-85e0-0fd247eb1dae admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:22.027645 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16c8a04-337e-4935-85e0-0fd247eb1dae admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:22.027918 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16c8a04-337e-4935-85e0-0fd247eb1dae admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:22.028603 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16c8a04-337e-4935-85e0-0fd247eb1dae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:22.030568 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16c8a04-337e-4935-85e0-0fd247eb1dae admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:22.157766 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1338/2711] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:33:22 2022] POST /identity/v3/projects => generated 387 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:33:22.165682 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab53472-8d1b-4a8f-898f-0720ea9f2bac admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:22.178602 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab53472-8d1b-4a8f-898f-0720ea9f2bac admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:22.262511 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5838ac37-8222-43d6-8891-29682d92bcaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:22.263490 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5838ac37-8222-43d6-8891-29682d92bcaa admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:22.263963 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5838ac37-8222-43d6-8891-29682d92bcaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:22.264583 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5838ac37-8222-43d6-8891-29682d92bcaa admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:22.265476 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5838ac37-8222-43d6-8891-29682d92bcaa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:22.267997 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5838ac37-8222-43d6-8891-29682d92bcaa admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:22.363411 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1374/2712] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:33:22 2022] POST /identity/v3/users => generated 294 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:33:22.367736 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16c8a04-337e-4935-85e0-0fd247eb1dae admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:22.379859 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16c8a04-337e-4935-85e0-0fd247eb1dae admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:22.385445 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fe7d23-3ee9-45ff-afcd-0f2f82cda1c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:22.386048 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fe7d23-3ee9-45ff-afcd-0f2f82cda1c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:22.386319 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fe7d23-3ee9-45ff-afcd-0f2f82cda1c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:22.386588 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fe7d23-3ee9-45ff-afcd-0f2f82cda1c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:22.387276 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fe7d23-3ee9-45ff-afcd-0f2f82cda1c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:22.389190 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fe7d23-3ee9-45ff-afcd-0f2f82cda1c1 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:22.403461 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1339/2713] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:33:22 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:33:22.407753 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5838ac37-8222-43d6-8891-29682d92bcaa admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:22.421110 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5838ac37-8222-43d6-8891-29682d92bcaa admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:22.426775 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46988796-7ed8-48a5-b58e-ea00832a482c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:22.427593 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46988796-7ed8-48a5-b58e-ea00832a482c admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:22.428244 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46988796-7ed8-48a5-b58e-ea00832a482c admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:22.428542 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46988796-7ed8-48a5-b58e-ea00832a482c admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:22.429509 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46988796-7ed8-48a5-b58e-ea00832a482c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:22.430897 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46988796-7ed8-48a5-b58e-ea00832a482c admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:22.448820 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1375/2714] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:33:22 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:33:22.453113 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fe7d23-3ee9-45ff-afcd-0f2f82cda1c1 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:22.465304 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fe7d23-3ee9-45ff-afcd-0f2f82cda1c1 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:22.471194 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0747478-1b21-4144-a6ad-8ddcc949a66b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:22.495352 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0747478-1b21-4144-a6ad-8ddcc949a66b admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:22.495352 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0747478-1b21-4144-a6ad-8ddcc949a66b admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:22.495352 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0747478-1b21-4144-a6ad-8ddcc949a66b admin admin] PATH_INFO: `/v3/projects/354d7e734abc4f5386526ce5fea7e3ae/users/8f87796bcdda422aa8307608bc0074e5/roles/8feebc60363c4feb870718b1721e5324` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:22.530448 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0747478-1b21-4144-a6ad-8ddcc949a66b admin admin] RBAC: Authorizing `identity:create_grant(project_id=354d7e734abc4f5386526ce5fea7e3ae, user_id=8f87796bcdda422aa8307608bc0074e5, role_id=8feebc60363c4feb870718b1721e5324)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:22.537118 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0747478-1b21-4144-a6ad-8ddcc949a66b admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:22.539947 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:22.643010 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1340/2715] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:33:22 2022] PUT /identity/v3/projects/354d7e734abc4f5386526ce5fea7e3ae/users/8f87796bcdda422aa8307608bc0074e5/roles/8feebc60363c4feb870718b1721e5324 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:33:22.646906 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5be0e4-1ab3-4130-8e5e-03cb5295fdfa None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:22.648882 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5be0e4-1ab3-4130-8e5e-03cb5295fdfa None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:22.649763 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5be0e4-1ab3-4130-8e5e-03cb5295fdfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:22.727681 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-2c5be0e4-1ab3-4130-8e5e-03cb5295fdfa None None] MFA Rules not processed for user `8f87796bcdda422aa8307608bc0074e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:33:22.727681 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-2c5be0e4-1ab3-4130-8e5e-03cb5295fdfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:22.825393 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1376/2716] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:33:22 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:33:22.830474 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0747478-1b21-4144-a6ad-8ddcc949a66b admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:22.919549 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0747478-1b21-4144-a6ad-8ddcc949a66b admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:22.924910 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2ed806-8e49-4ced-93b7-625cf7087cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:22.926720 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2ed806-8e49-4ced-93b7-625cf7087cdc admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:22.926899 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2ed806-8e49-4ced-93b7-625cf7087cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:22.927049 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2ed806-8e49-4ced-93b7-625cf7087cdc admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:22.927907 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2ed806-8e49-4ced-93b7-625cf7087cdc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:22.930246 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2ed806-8e49-4ced-93b7-625cf7087cdc admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:23.065246 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1341/2717] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:33:22 2022] POST /identity/v3/projects => generated 387 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:33:23.068841 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5be0e4-1ab3-4130-8e5e-03cb5295fdfa None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:23.085955 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5be0e4-1ab3-4130-8e5e-03cb5295fdfa None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:23.094977 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:23.147208 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362e6b46-026c-4380-9d85-cfe6177d9cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:23.147873 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362e6b46-026c-4380-9d85-cfe6177d9cd5 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:23.148201 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362e6b46-026c-4380-9d85-cfe6177d9cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:23.148480 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362e6b46-026c-4380-9d85-cfe6177d9cd5 admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:23.149213 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362e6b46-026c-4380-9d85-cfe6177d9cd5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:23.151448 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362e6b46-026c-4380-9d85-cfe6177d9cd5 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:23.238915 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1377/2718] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:33:23 2022] POST /identity/v3/users => generated 300 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:33:23.243307 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2ed806-8e49-4ced-93b7-625cf7087cdc admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:23.256463 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2ed806-8e49-4ced-93b7-625cf7087cdc admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:23.261976 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6acfb9-017c-4c79-8f31-6d0da41fdf74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:23.262599 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6acfb9-017c-4c79-8f31-6d0da41fdf74 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:23.262887 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6acfb9-017c-4c79-8f31-6d0da41fdf74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:23.263166 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6acfb9-017c-4c79-8f31-6d0da41fdf74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:23.263909 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6acfb9-017c-4c79-8f31-6d0da41fdf74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:23.265554 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6acfb9-017c-4c79-8f31-6d0da41fdf74 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:23.276835 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1342/2719] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:33:23 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:33:23.282224 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362e6b46-026c-4380-9d85-cfe6177d9cd5 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:23.294249 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362e6b46-026c-4380-9d85-cfe6177d9cd5 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:23.301438 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51dabb0-88a1-4690-9399-8fe8994a6c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:23.302136 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51dabb0-88a1-4690-9399-8fe8994a6c16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:23.302426 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51dabb0-88a1-4690-9399-8fe8994a6c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:23.302777 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51dabb0-88a1-4690-9399-8fe8994a6c16 admin admin] PATH_INFO: `/v3/projects/f2a28266138a42faa8dc7567ccf0c604/users/1551235a9057413fac06241b5af82774/roles/a432dd4735944ecfab2fa5ce1705d356` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:23.321974 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51dabb0-88a1-4690-9399-8fe8994a6c16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2a28266138a42faa8dc7567ccf0c604, user_id=1551235a9057413fac06241b5af82774, role_id=a432dd4735944ecfab2fa5ce1705d356)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:23.324841 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51dabb0-88a1-4690-9399-8fe8994a6c16 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:23.404173 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1378/2720] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:33:23 2022] PUT /identity/v3/projects/f2a28266138a42faa8dc7567ccf0c604/users/1551235a9057413fac06241b5af82774/roles/a432dd4735944ecfab2fa5ce1705d356 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:33:23.408597 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6acfb9-017c-4c79-8f31-6d0da41fdf74 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:23.420174 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6acfb9-017c-4c79-8f31-6d0da41fdf74 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:23.484961 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366f2268-0a52-4105-a5db-3138b82d184b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:23.485651 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366f2268-0a52-4105-a5db-3138b82d184b admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:23.485931 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366f2268-0a52-4105-a5db-3138b82d184b admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:23.486199 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366f2268-0a52-4105-a5db-3138b82d184b admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:23.487225 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366f2268-0a52-4105-a5db-3138b82d184b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:23.488715 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366f2268-0a52-4105-a5db-3138b82d184b admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:23.499191 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1343/2721] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:33:23 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:33:23.503679 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51dabb0-88a1-4690-9399-8fe8994a6c16 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:23.516999 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51dabb0-88a1-4690-9399-8fe8994a6c16 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:23.523935 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099e3216-5cfe-46d5-ae78-73ac75a2e661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:23.524669 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e3216-5cfe-46d5-ae78-73ac75a2e661 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:23.524999 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e3216-5cfe-46d5-ae78-73ac75a2e661 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:23.525315 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e3216-5cfe-46d5-ae78-73ac75a2e661 admin admin] PATH_INFO: `/v3/projects/f2a28266138a42faa8dc7567ccf0c604/users/1551235a9057413fac06241b5af82774/roles/a432dd4735944ecfab2fa5ce1705d356` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:23.527710 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099e3216-5cfe-46d5-ae78-73ac75a2e661 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2a28266138a42faa8dc7567ccf0c604, user_id=1551235a9057413fac06241b5af82774, role_id=a432dd4735944ecfab2fa5ce1705d356)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:23.530422 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099e3216-5cfe-46d5-ae78-73ac75a2e661 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:23.542034 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1379/2722] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:33:23 2022] PUT /identity/v3/projects/f2a28266138a42faa8dc7567ccf0c604/users/1551235a9057413fac06241b5af82774/roles/a432dd4735944ecfab2fa5ce1705d356 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:33:23.552831 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47e2e1f-1e90-4b44-ac2f-b8c6a78cee96 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:23.553189 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47e2e1f-1e90-4b44-ac2f-b8c6a78cee96 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:23.553391 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47e2e1f-1e90-4b44-ac2f-b8c6a78cee96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:23.583186 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-b47e2e1f-1e90-4b44-ac2f-b8c6a78cee96 None None] MFA Rules not processed for user `1551235a9057413fac06241b5af82774`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:33:23.584567 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-b47e2e1f-1e90-4b44-ac2f-b8c6a78cee96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:23.649980 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:23.677614 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1344/2723] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:33:23 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:33:23.691010 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099e3216-5cfe-46d5-ae78-73ac75a2e661 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:23.693635 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-099e3216-5cfe-46d5-ae78-73ac75a2e661 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:23.763809 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099e3216-5cfe-46d5-ae78-73ac75a2e661 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:23.770305 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1290cd11-714b-4737-9b3f-7d4bb46c9bbd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e31f2728056b460fa474478eb85ca1d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:23.770305 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1290cd11-714b-4737-9b3f-7d4bb46c9bbd service cinder] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:23.770305 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1290cd11-714b-4737-9b3f-7d4bb46c9bbd service cinder] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:23.770305 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1290cd11-714b-4737-9b3f-7d4bb46c9bbd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:23.781673 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1290cd11-714b-4737-9b3f-7d4bb46c9bbd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:23.783091 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1290cd11-714b-4737-9b3f-7d4bb46c9bbd service cinder] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:23.794515 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1380/2724] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:33:23 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:33:23.897221 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47e2e1f-1e90-4b44-ac2f-b8c6a78cee96 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:24.031631 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47e2e1f-1e90-4b44-ac2f-b8c6a78cee96 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:24.040047 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4292bc3-5673-4e96-9501-bac96f32f76c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:24.040834 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4292bc3-5673-4e96-9501-bac96f32f76c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:24.041312 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4292bc3-5673-4e96-9501-bac96f32f76c admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:24.041578 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4292bc3-5673-4e96-9501-bac96f32f76c admin admin] PATH_INFO: `/v3/users/8f87796bcdda422aa8307608bc0074e5` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:24.042981 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4292bc3-5673-4e96-9501-bac96f32f76c admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f87796bcdda422aa8307608bc0074e5)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:24.045213 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4292bc3-5673-4e96-9501-bac96f32f76c admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:24.284640 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c4292bc3-5673-4e96-9501-bac96f32f76c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f87796bcdda422aa8307608bc0074e5'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:24.302481 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c4292bc3-5673-4e96-9501-bac96f32f76c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f87796bcdda422aa8307608bc0074e5'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:24.303242 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c4292bc3-5673-4e96-9501-bac96f32f76c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f87796bcdda422aa8307608bc0074e5'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:24.310531 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c4292bc3-5673-4e96-9501-bac96f32f76c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f87796bcdda422aa8307608bc0074e5'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:24.311218 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c4292bc3-5673-4e96-9501-bac96f32f76c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f87796bcdda422aa8307608bc0074e5'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:24.396004 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1345/2725] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:33:23 2022] DELETE /identity/v3/users/8f87796bcdda422aa8307608bc0074e5 => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:33:24.401250 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a1087b-3b89-4f97-b340-01eabf575ba0 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:24.401828 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a1087b-3b89-4f97-b340-01eabf575ba0 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:24.401936 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a1087b-3b89-4f97-b340-01eabf575ba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:24.428217 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-31a1087b-3b89-4f97-b340-01eabf575ba0 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:33:24.430073 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-31a1087b-3b89-4f97-b340-01eabf575ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:24.533652 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1381/2726] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:33:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:33:24.545704 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4292bc3-5673-4e96-9501-bac96f32f76c admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:24.549461 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-c4292bc3-5673-4e96-9501-bac96f32f76c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:24.632103 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4292bc3-5673-4e96-9501-bac96f32f76c admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:24.637439 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03d427d-bc5c-4da8-8ff9-f59de9fb2a69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:24.638077 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03d427d-bc5c-4da8-8ff9-f59de9fb2a69 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:24.638363 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03d427d-bc5c-4da8-8ff9-f59de9fb2a69 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:24.638639 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03d427d-bc5c-4da8-8ff9-f59de9fb2a69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:24.650915 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03d427d-bc5c-4da8-8ff9-f59de9fb2a69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:24.652525 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03d427d-bc5c-4da8-8ff9-f59de9fb2a69 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:24.662993 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1346/2727] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:33:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:33:25.113978 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a1087b-3b89-4f97-b340-01eabf575ba0 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:25.118246 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-31a1087b-3b89-4f97-b340-01eabf575ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:25.135520 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a1087b-3b89-4f97-b340-01eabf575ba0 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:25.144388 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea7e12d-5d49-49a1-9f57-45d0a6066e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:25.144388 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea7e12d-5d49-49a1-9f57-45d0a6066e78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:25.144388 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea7e12d-5d49-49a1-9f57-45d0a6066e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:25.144388 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea7e12d-5d49-49a1-9f57-45d0a6066e78 admin admin] PATH_INFO: `/v3/projects/354d7e734abc4f5386526ce5fea7e3ae` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:25.145423 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea7e12d-5d49-49a1-9f57-45d0a6066e78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=354d7e734abc4f5386526ce5fea7e3ae)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:25.147286 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea7e12d-5d49-49a1-9f57-45d0a6066e78 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:25.149119 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:25.308012 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-cea7e12d-5d49-49a1-9f57-45d0a6066e78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '354d7e734abc4f5386526ce5fea7e3ae', 'request_id': 'req-cea7e12d-5d49-49a1-9f57-45d0a6066e78'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:25.317244 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-cea7e12d-5d49-49a1-9f57-45d0a6066e78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '354d7e734abc4f5386526ce5fea7e3ae', 'request_id': 'req-cea7e12d-5d49-49a1-9f57-45d0a6066e78'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:25.442537 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-cea7e12d-5d49-49a1-9f57-45d0a6066e78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '354d7e734abc4f5386526ce5fea7e3ae', 'request_id': 'req-cea7e12d-5d49-49a1-9f57-45d0a6066e78'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:25.457232 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-cea7e12d-5d49-49a1-9f57-45d0a6066e78 admin admin] The token cache is being invalidate because project 354d7e734abc4f5386526ce5fea7e3ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:33:25.457521 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-cea7e12d-5d49-49a1-9f57-45d0a6066e78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:25.458230 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-cea7e12d-5d49-49a1-9f57-45d0a6066e78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:25.463659 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1382/2728] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:33:25 2022] DELETE /identity/v3/projects/354d7e734abc4f5386526ce5fea7e3ae => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:33:25.466287 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea7e12d-5d49-49a1-9f57-45d0a6066e78 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:25.466430 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03d427d-bc5c-4da8-8ff9-f59de9fb2a69 service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:25.467871 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-e03d427d-bc5c-4da8-8ff9-f59de9fb2a69 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:25.468137 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-cea7e12d-5d49-49a1-9f57-45d0a6066e78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:25.481793 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03d427d-bc5c-4da8-8ff9-f59de9fb2a69 service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:25.491451 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:25.542887 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581fdfcd-9eac-4fd2-9122-6f7c309b2297 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:25.543907 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581fdfcd-9eac-4fd2-9122-6f7c309b2297 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:25.550344 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581fdfcd-9eac-4fd2-9122-6f7c309b2297 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:25.550344 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581fdfcd-9eac-4fd2-9122-6f7c309b2297 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:25.550344 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-581fdfcd-9eac-4fd2-9122-6f7c309b2297 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:25.551014 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea7e12d-5d49-49a1-9f57-45d0a6066e78 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:25.557922 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d482391-28e3-401d-a91b-83815658aba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:25.558676 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d482391-28e3-401d-a91b-83815658aba5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:25.559060 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d482391-28e3-401d-a91b-83815658aba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:25.559475 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d482391-28e3-401d-a91b-83815658aba5 admin admin] PATH_INFO: `/v3/users/1551235a9057413fac06241b5af82774` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:25.560873 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d482391-28e3-401d-a91b-83815658aba5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1551235a9057413fac06241b5af82774)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:25.562911 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d482391-28e3-401d-a91b-83815658aba5 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:25.624109 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581fdfcd-9eac-4fd2-9122-6f7c309b2297 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:25.625654 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581fdfcd-9eac-4fd2-9122-6f7c309b2297 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:25.636996 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1347/2729] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:33:25 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:33:25.807730 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-2d482391-28e3-401d-a91b-83815658aba5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1551235a9057413fac06241b5af82774'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:25.833347 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-2d482391-28e3-401d-a91b-83815658aba5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1551235a9057413fac06241b5af82774'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:25.834102 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-2d482391-28e3-401d-a91b-83815658aba5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1551235a9057413fac06241b5af82774'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:25.834756 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-2d482391-28e3-401d-a91b-83815658aba5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1551235a9057413fac06241b5af82774'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:25.905891 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-2d482391-28e3-401d-a91b-83815658aba5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1551235a9057413fac06241b5af82774'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:25.922393 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1383/2730] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:33:25 2022] DELETE /identity/v3/users/1551235a9057413fac06241b5af82774 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:33:26.560311 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581fdfcd-9eac-4fd2-9122-6f7c309b2297 service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:26.581996 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581fdfcd-9eac-4fd2-9122-6f7c309b2297 service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:26.661608 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7382c4-a26a-4ad5-ab9a-8b03e3bf2757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:26.662280 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7382c4-a26a-4ad5-ab9a-8b03e3bf2757 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:26.662568 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7382c4-a26a-4ad5-ab9a-8b03e3bf2757 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:26.662849 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7382c4-a26a-4ad5-ab9a-8b03e3bf2757 admin admin] PATH_INFO: `/v3/projects/f2a28266138a42faa8dc7567ccf0c604` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:26.664147 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7382c4-a26a-4ad5-ab9a-8b03e3bf2757 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2a28266138a42faa8dc7567ccf0c604)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:26.666103 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7382c4-a26a-4ad5-ab9a-8b03e3bf2757 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:26.764039 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-8d7382c4-a26a-4ad5-ab9a-8b03e3bf2757 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2a28266138a42faa8dc7567ccf0c604', 'request_id': 'req-8d7382c4-a26a-4ad5-ab9a-8b03e3bf2757'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:26.773147 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-8d7382c4-a26a-4ad5-ab9a-8b03e3bf2757 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2a28266138a42faa8dc7567ccf0c604', 'request_id': 'req-8d7382c4-a26a-4ad5-ab9a-8b03e3bf2757'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:26.779764 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:26.881267 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-8d7382c4-a26a-4ad5-ab9a-8b03e3bf2757 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2a28266138a42faa8dc7567ccf0c604', 'request_id': 'req-8d7382c4-a26a-4ad5-ab9a-8b03e3bf2757'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:26.897019 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-8d7382c4-a26a-4ad5-ab9a-8b03e3bf2757 admin admin] The token cache is being invalidate because project f2a28266138a42faa8dc7567ccf0c604 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:33:26.897497 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-8d7382c4-a26a-4ad5-ab9a-8b03e3bf2757 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:26.898277 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-8d7382c4-a26a-4ad5-ab9a-8b03e3bf2757 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:26.900461 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1348/2731] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:33:26 2022] DELETE /identity/v3/projects/f2a28266138a42faa8dc7567ccf0c604 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:33:26.905374 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d5c6a-4ac2-423e-b505-4426bdd69324 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:26.907131 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d5c6a-4ac2-423e-b505-4426bdd69324 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:26.907131 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d5c6a-4ac2-423e-b505-4426bdd69324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:26.928520 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-b64d5c6a-4ac2-423e-b505-4426bdd69324 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:33:26.935955 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-b64d5c6a-4ac2-423e-b505-4426bdd69324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:26.935955 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:27.004418 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1384/2732] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:33:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:33:27.012743 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2a9697-b6c1-44cb-ba6c-452bc1e3eace None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:27.013116 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2a9697-b6c1-44cb-ba6c-452bc1e3eace None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:27.013441 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2a9697-b6c1-44cb-ba6c-452bc1e3eace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:27.034592 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-7e2a9697-b6c1-44cb-ba6c-452bc1e3eace None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:33:27.036277 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-7e2a9697-b6c1-44cb-ba6c-452bc1e3eace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:27.044350 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1349/2733] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:33:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:33:27.048754 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64d5c6a-4ac2-423e-b505-4426bdd69324 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:27.060384 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64d5c6a-4ac2-423e-b505-4426bdd69324 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:27.066231 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11375682-be53-41e5-aeb9-b8ea59de58a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:27.066980 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11375682-be53-41e5-aeb9-b8ea59de58a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:27.067270 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11375682-be53-41e5-aeb9-b8ea59de58a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:27.067581 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11375682-be53-41e5-aeb9-b8ea59de58a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:27.068281 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11375682-be53-41e5-aeb9-b8ea59de58a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:33:27.068586 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11375682-be53-41e5-aeb9-b8ea59de58a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:27.070323 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11375682-be53-41e5-aeb9-b8ea59de58a3 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:27.083648 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1385/2734] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Aug 4 21:33:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:33:27.088182 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2a9697-b6c1-44cb-ba6c-452bc1e3eace None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:27.098847 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2a9697-b6c1-44cb-ba6c-452bc1e3eace None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:27.104322 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76051d3e-6cd9-4104-ab5b-09f07a62ad34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:27.105051 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76051d3e-6cd9-4104-ab5b-09f07a62ad34 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:27.105551 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76051d3e-6cd9-4104-ab5b-09f07a62ad34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:27.105925 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76051d3e-6cd9-4104-ab5b-09f07a62ad34 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:27.106608 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76051d3e-6cd9-4104-ab5b-09f07a62ad34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:27.108641 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76051d3e-6cd9-4104-ab5b-09f07a62ad34 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:27.207695 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1350/2735] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:33:27 2022] POST /identity/v3/projects => generated 385 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:33:27.213029 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11375682-be53-41e5-aeb9-b8ea59de58a3 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:27.228105 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11375682-be53-41e5-aeb9-b8ea59de58a3 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:27.296763 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53edc706-a2e6-4206-85c1-7314288dff61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:27.297258 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53edc706-a2e6-4206-85c1-7314288dff61 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:27.297543 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53edc706-a2e6-4206-85c1-7314288dff61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:27.297821 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53edc706-a2e6-4206-85c1-7314288dff61 admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:27.298495 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53edc706-a2e6-4206-85c1-7314288dff61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:27.301138 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53edc706-a2e6-4206-85c1-7314288dff61 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:27.409913 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1386/2736] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:33:27 2022] POST /identity/v3/users => generated 293 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:33:27.418149 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76051d3e-6cd9-4104-ab5b-09f07a62ad34 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:27.430593 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76051d3e-6cd9-4104-ab5b-09f07a62ad34 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:27.436846 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e76f46-9b3c-4a84-9044-34bfa13ea267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:27.437761 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e76f46-9b3c-4a84-9044-34bfa13ea267 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:27.438214 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e76f46-9b3c-4a84-9044-34bfa13ea267 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:27.438661 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e76f46-9b3c-4a84-9044-34bfa13ea267 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:27.440214 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e76f46-9b3c-4a84-9044-34bfa13ea267 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:27.461110 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e76f46-9b3c-4a84-9044-34bfa13ea267 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:27.464820 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1351/2737] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:33:27 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:33:27.469810 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53edc706-a2e6-4206-85c1-7314288dff61 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:27.485508 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53edc706-a2e6-4206-85c1-7314288dff61 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:27.495507 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c76512-b95d-4670-b39c-e43f5415fd9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:27.496226 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c76512-b95d-4670-b39c-e43f5415fd9b admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:27.496706 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c76512-b95d-4670-b39c-e43f5415fd9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:27.497138 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c76512-b95d-4670-b39c-e43f5415fd9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:27.498267 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c76512-b95d-4670-b39c-e43f5415fd9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:27.501779 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c76512-b95d-4670-b39c-e43f5415fd9b admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:27.527595 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1387/2738] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:33:27 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:33:27.533036 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e76f46-9b3c-4a84-9044-34bfa13ea267 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:27.548412 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e76f46-9b3c-4a84-9044-34bfa13ea267 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:27.557349 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181f4124-9a33-45ab-99b5-c98daf143ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:27.561239 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181f4124-9a33-45ab-99b5-c98daf143ebd admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:27.561239 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181f4124-9a33-45ab-99b5-c98daf143ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:27.561239 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181f4124-9a33-45ab-99b5-c98daf143ebd admin admin] PATH_INFO: `/v3/projects/612ab67992124f61934e96912f1cce3c/users/078b4076a64a4c87a11f86d85c815d36/roles/8feebc60363c4feb870718b1721e5324` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:27.577129 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181f4124-9a33-45ab-99b5-c98daf143ebd admin admin] RBAC: Authorizing `identity:create_grant(project_id=612ab67992124f61934e96912f1cce3c, user_id=078b4076a64a4c87a11f86d85c815d36, role_id=8feebc60363c4feb870718b1721e5324)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:27.583881 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181f4124-9a33-45ab-99b5-c98daf143ebd admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:27.679805 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1352/2739] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:33:27 2022] PUT /identity/v3/projects/612ab67992124f61934e96912f1cce3c/users/078b4076a64a4c87a11f86d85c815d36/roles/8feebc60363c4feb870718b1721e5324 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:33:27.691922 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbb8c69-5103-434e-931c-cc0b740b20d9 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:27.692250 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbb8c69-5103-434e-931c-cc0b740b20d9 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:27.692620 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbb8c69-5103-434e-931c-cc0b740b20d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:27.725744 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-ddbb8c69-5103-434e-931c-cc0b740b20d9 None None] MFA Rules not processed for user `078b4076a64a4c87a11f86d85c815d36`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:33:27.727015 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-ddbb8c69-5103-434e-931c-cc0b740b20d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:27.805745 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1388/2740] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:33:27 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:33:27.810340 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181f4124-9a33-45ab-99b5-c98daf143ebd admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:27.888028 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181f4124-9a33-45ab-99b5-c98daf143ebd admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:27.897028 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b906d4-b757-48e7-bf39-a56c53823254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:27.898100 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b906d4-b757-48e7-bf39-a56c53823254 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:27.898613 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b906d4-b757-48e7-bf39-a56c53823254 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:27.899111 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b906d4-b757-48e7-bf39-a56c53823254 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:27.900399 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b906d4-b757-48e7-bf39-a56c53823254 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:27.904530 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b906d4-b757-48e7-bf39-a56c53823254 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:28.044417 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1353/2741] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:33:27 2022] POST /identity/v3/projects => generated 387 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:33:28.045652 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbb8c69-5103-434e-931c-cc0b740b20d9 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:28.053077 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:28.063557 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbb8c69-5103-434e-931c-cc0b740b20d9 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:28.163750 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d31a2a-e1ad-45f4-8abf-e676e517da69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:28.164570 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d31a2a-e1ad-45f4-8abf-e676e517da69 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:28.164934 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d31a2a-e1ad-45f4-8abf-e676e517da69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:28.165209 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d31a2a-e1ad-45f4-8abf-e676e517da69 admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:28.165892 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d31a2a-e1ad-45f4-8abf-e676e517da69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:28.168061 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d31a2a-e1ad-45f4-8abf-e676e517da69 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:28.287350 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1389/2742] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:33:28 2022] POST /identity/v3/users => generated 294 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:33:28.293340 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b906d4-b757-48e7-bf39-a56c53823254 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:28.307561 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b906d4-b757-48e7-bf39-a56c53823254 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:28.312923 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b1ced9-6c0d-46f9-ae5b-121dfff6d03a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:28.313748 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b1ced9-6c0d-46f9-ae5b-121dfff6d03a admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:28.314064 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b1ced9-6c0d-46f9-ae5b-121dfff6d03a admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:28.314339 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b1ced9-6c0d-46f9-ae5b-121dfff6d03a admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:28.315083 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b1ced9-6c0d-46f9-ae5b-121dfff6d03a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:28.316848 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b1ced9-6c0d-46f9-ae5b-121dfff6d03a admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:28.328990 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1354/2743] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:33:28 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:33:28.333558 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d31a2a-e1ad-45f4-8abf-e676e517da69 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:28.344637 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d31a2a-e1ad-45f4-8abf-e676e517da69 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:28.351080 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a96942-f0ba-4d07-b1cb-dfefee562e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:28.352454 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a96942-f0ba-4d07-b1cb-dfefee562e9e admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:28.352454 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a96942-f0ba-4d07-b1cb-dfefee562e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:28.352454 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a96942-f0ba-4d07-b1cb-dfefee562e9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:28.353113 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a96942-f0ba-4d07-b1cb-dfefee562e9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:28.355241 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a96942-f0ba-4d07-b1cb-dfefee562e9e admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:28.373250 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1390/2744] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:33:28 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:33:28.374711 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b1ced9-6c0d-46f9-ae5b-121dfff6d03a admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:28.392497 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b1ced9-6c0d-46f9-ae5b-121dfff6d03a admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:28.401008 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bef35a8-bc54-4ff8-acf3-41e02f1eade2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:28.402127 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bef35a8-bc54-4ff8-acf3-41e02f1eade2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:28.402611 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bef35a8-bc54-4ff8-acf3-41e02f1eade2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:28.403197 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bef35a8-bc54-4ff8-acf3-41e02f1eade2 admin admin] PATH_INFO: `/v3/projects/69f75d920fbf47d99dfe6e49349b0637/users/6a70b0a2066b4f02a92e9c0a99ec2ff8/roles/8feebc60363c4feb870718b1721e5324` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:28.444645 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bef35a8-bc54-4ff8-acf3-41e02f1eade2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69f75d920fbf47d99dfe6e49349b0637, user_id=6a70b0a2066b4f02a92e9c0a99ec2ff8, role_id=8feebc60363c4feb870718b1721e5324)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:28.448701 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bef35a8-bc54-4ff8-acf3-41e02f1eade2 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:28.532928 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1355/2745] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:33:28 2022] PUT /identity/v3/projects/69f75d920fbf47d99dfe6e49349b0637/users/6a70b0a2066b4f02a92e9c0a99ec2ff8/roles/8feebc60363c4feb870718b1721e5324 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:33:28.547489 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bca4f7-871e-46da-81e6-3cf628a2ef4e None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:28.547948 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bca4f7-871e-46da-81e6-3cf628a2ef4e None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:28.548398 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bca4f7-871e-46da-81e6-3cf628a2ef4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:28.595507 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-43bca4f7-871e-46da-81e6-3cf628a2ef4e None None] MFA Rules not processed for user `6a70b0a2066b4f02a92e9c0a99ec2ff8`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:33:28.597775 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-43bca4f7-871e-46da-81e6-3cf628a2ef4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:28.704941 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1391/2746] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:33:28 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:33:28.706687 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bef35a8-bc54-4ff8-acf3-41e02f1eade2 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:28.794375 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:28.838385 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bef35a8-bc54-4ff8-acf3-41e02f1eade2 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:28.849485 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8a3728-0cbf-480e-a5bd-cc9963f3f474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:28.853651 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8a3728-0cbf-480e-a5bd-cc9963f3f474 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:28.853651 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8a3728-0cbf-480e-a5bd-cc9963f3f474 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:28.853651 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8a3728-0cbf-480e-a5bd-cc9963f3f474 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:28.853651 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8a3728-0cbf-480e-a5bd-cc9963f3f474 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:28.859103 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8a3728-0cbf-480e-a5bd-cc9963f3f474 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:28.976436 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1356/2747] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:33:28 2022] POST /identity/v3/projects => generated 385 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:33:28.978203 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bca4f7-871e-46da-81e6-3cf628a2ef4e None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:29.003052 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bca4f7-871e-46da-81e6-3cf628a2ef4e None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:29.097254 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1316a265-3a74-409c-a0b5-2071f16283e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:29.097892 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1316a265-3a74-409c-a0b5-2071f16283e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:29.098173 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1316a265-3a74-409c-a0b5-2071f16283e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:29.098603 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1316a265-3a74-409c-a0b5-2071f16283e9 admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:29.099264 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1316a265-3a74-409c-a0b5-2071f16283e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:29.101268 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1316a265-3a74-409c-a0b5-2071f16283e9 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:29.211484 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1392/2748] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:33:28 2022] POST /identity/v3/users => generated 299 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:33:29.213080 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8a3728-0cbf-480e-a5bd-cc9963f3f474 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:29.231992 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8a3728-0cbf-480e-a5bd-cc9963f3f474 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:29.237129 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1b21ea-56f6-4d91-af94-839c9fbf943b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:29.241062 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1b21ea-56f6-4d91-af94-839c9fbf943b admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:29.241062 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1b21ea-56f6-4d91-af94-839c9fbf943b admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:29.241062 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1b21ea-56f6-4d91-af94-839c9fbf943b admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:29.241062 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1b21ea-56f6-4d91-af94-839c9fbf943b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:29.242413 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1b21ea-56f6-4d91-af94-839c9fbf943b admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:29.259036 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1357/2749] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:33:29 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:33:29.264205 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1316a265-3a74-409c-a0b5-2071f16283e9 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:29.273795 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:29.284652 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1316a265-3a74-409c-a0b5-2071f16283e9 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:29.292734 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcdf665-3783-4edc-8102-5a79a928a08e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:29.293885 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcdf665-3783-4edc-8102-5a79a928a08e admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:29.294382 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcdf665-3783-4edc-8102-5a79a928a08e admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:29.294882 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcdf665-3783-4edc-8102-5a79a928a08e admin admin] PATH_INFO: `/v3/projects/4625c6b2d0c54e90a43bbeda3c027792/users/d96dd5eafc7341c8b9ff6b80ca01cdd4/roles/a432dd4735944ecfab2fa5ce1705d356` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:29.324561 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcdf665-3783-4edc-8102-5a79a928a08e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4625c6b2d0c54e90a43bbeda3c027792, user_id=d96dd5eafc7341c8b9ff6b80ca01cdd4, role_id=a432dd4735944ecfab2fa5ce1705d356)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:29.332248 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcdf665-3783-4edc-8102-5a79a928a08e admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:29.437450 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1393/2750] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:33:29 2022] PUT /identity/v3/projects/4625c6b2d0c54e90a43bbeda3c027792/users/d96dd5eafc7341c8b9ff6b80ca01cdd4/roles/a432dd4735944ecfab2fa5ce1705d356 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:33:29.443213 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1b21ea-56f6-4d91-af94-839c9fbf943b admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:29.452868 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1b21ea-56f6-4d91-af94-839c9fbf943b admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:29.520301 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1735169-7812-498d-bdd6-dc23d94a9dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:29.520917 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1735169-7812-498d-bdd6-dc23d94a9dec admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:29.521824 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1735169-7812-498d-bdd6-dc23d94a9dec admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:29.521824 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1735169-7812-498d-bdd6-dc23d94a9dec admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:29.522170 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1735169-7812-498d-bdd6-dc23d94a9dec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:29.523911 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1735169-7812-498d-bdd6-dc23d94a9dec admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:29.539959 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1358/2751] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:33:29 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:33:29.544227 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcdf665-3783-4edc-8102-5a79a928a08e admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:29.557625 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcdf665-3783-4edc-8102-5a79a928a08e admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:29.562400 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c55656-9a19-4bb0-bc73-811a8988c4c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:29.563081 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c55656-9a19-4bb0-bc73-811a8988c4c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:29.563388 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c55656-9a19-4bb0-bc73-811a8988c4c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:29.563666 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c55656-9a19-4bb0-bc73-811a8988c4c4 admin admin] PATH_INFO: `/v3/projects/4625c6b2d0c54e90a43bbeda3c027792/users/d96dd5eafc7341c8b9ff6b80ca01cdd4/roles/a432dd4735944ecfab2fa5ce1705d356` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:29.565878 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c55656-9a19-4bb0-bc73-811a8988c4c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4625c6b2d0c54e90a43bbeda3c027792, user_id=d96dd5eafc7341c8b9ff6b80ca01cdd4, role_id=a432dd4735944ecfab2fa5ce1705d356)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:29.568793 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c55656-9a19-4bb0-bc73-811a8988c4c4 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:29.582458 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1394/2752] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:33:29 2022] PUT /identity/v3/projects/4625c6b2d0c54e90a43bbeda3c027792/users/d96dd5eafc7341c8b9ff6b80ca01cdd4/roles/a432dd4735944ecfab2fa5ce1705d356 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:33:29.651221 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aef87e5-7fa5-4064-a37b-0e9dccc97414 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:29.651604 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aef87e5-7fa5-4064-a37b-0e9dccc97414 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:29.652005 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aef87e5-7fa5-4064-a37b-0e9dccc97414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:29.682028 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-6aef87e5-7fa5-4064-a37b-0e9dccc97414 None None] MFA Rules not processed for user `d96dd5eafc7341c8b9ff6b80ca01cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:33:29.683243 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-6aef87e5-7fa5-4064-a37b-0e9dccc97414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:29.756598 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1359/2753] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:33:29 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:33:29.767795 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c55656-9a19-4bb0-bc73-811a8988c4c4 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:29.770013 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-81c55656-9a19-4bb0-bc73-811a8988c4c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:29.837431 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c55656-9a19-4bb0-bc73-811a8988c4c4 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:29.841945 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e96aed3-c77c-4df3-b897-d329b4bc0815 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e31f2728056b460fa474478eb85ca1d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:29.842684 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e96aed3-c77c-4df3-b897-d329b4bc0815 service cinder] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:29.843031 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e96aed3-c77c-4df3-b897-d329b4bc0815 service cinder] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:29.843417 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e96aed3-c77c-4df3-b897-d329b4bc0815 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:29.853590 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e96aed3-c77c-4df3-b897-d329b4bc0815 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:29.855032 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e96aed3-c77c-4df3-b897-d329b4bc0815 service cinder] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:29.864824 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1395/2754] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:33:29 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:33:31.093246 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aef87e5-7fa5-4064-a37b-0e9dccc97414 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:31.103940 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aef87e5-7fa5-4064-a37b-0e9dccc97414 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:31.107494 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55954677-9218-4bca-808f-6bcffe410cb1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e31f2728056b460fa474478eb85ca1d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:31.108109 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55954677-9218-4bca-808f-6bcffe410cb1 service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:31.108387 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55954677-9218-4bca-808f-6bcffe410cb1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:31.108659 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55954677-9218-4bca-808f-6bcffe410cb1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:31.141166 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:31.173543 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55954677-9218-4bca-808f-6bcffe410cb1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:31.175695 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55954677-9218-4bca-808f-6bcffe410cb1 service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:31.186854 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1360/2755] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:33:31 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:33:32.484020 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e96aed3-c77c-4df3-b897-d329b4bc0815 service cinder] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:32.485571 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-9e96aed3-c77c-4df3-b897-d329b4bc0815 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:32.564857 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e96aed3-c77c-4df3-b897-d329b4bc0815 service cinder] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:32.572981 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7a7321-8938-439f-adad-50184c3b5c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:32.574028 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7a7321-8938-439f-adad-50184c3b5c6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:32.574491 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7a7321-8938-439f-adad-50184c3b5c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:32.574987 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7a7321-8938-439f-adad-50184c3b5c6b admin admin] PATH_INFO: `/v3/users/2aab5b8821ca41509a424d73cd322453` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:32.576810 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7a7321-8938-439f-adad-50184c3b5c6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2aab5b8821ca41509a424d73cd322453)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:32.578919 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7a7321-8938-439f-adad-50184c3b5c6b admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:32.794909 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-5a7a7321-8938-439f-adad-50184c3b5c6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2aab5b8821ca41509a424d73cd322453'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:32.809352 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-5a7a7321-8938-439f-adad-50184c3b5c6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2aab5b8821ca41509a424d73cd322453'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:32.810044 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-5a7a7321-8938-439f-adad-50184c3b5c6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2aab5b8821ca41509a424d73cd322453'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:32.811620 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-5a7a7321-8938-439f-adad-50184c3b5c6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2aab5b8821ca41509a424d73cd322453'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:32.818276 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:32.900785 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-5a7a7321-8938-439f-adad-50184c3b5c6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2aab5b8821ca41509a424d73cd322453'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:32.916135 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1396/2756] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:33:32 2022] DELETE /identity/v3/users/2aab5b8821ca41509a424d73cd322453 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:33:32.921017 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9907ef1d-d358-4813-89ff-40846dd8092c None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:32.921555 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9907ef1d-d358-4813-89ff-40846dd8092c None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:32.921555 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9907ef1d-d358-4813-89ff-40846dd8092c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:32.942465 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-9907ef1d-d358-4813-89ff-40846dd8092c None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:33:32.944373 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-9907ef1d-d358-4813-89ff-40846dd8092c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:33.031151 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1361/2757] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:33:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:33:33.042080 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7a7321-8938-439f-adad-50184c3b5c6b admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:33.043996 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-5a7a7321-8938-439f-adad-50184c3b5c6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:33.116251 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7a7321-8938-439f-adad-50184c3b5c6b admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:33.121571 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8d2538-cd4b-4c75-8183-304e82cffe56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:33.122324 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8d2538-cd4b-4c75-8183-304e82cffe56 service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:33.122554 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8d2538-cd4b-4c75-8183-304e82cffe56 service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:33.122784 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8d2538-cd4b-4c75-8183-304e82cffe56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:33.133823 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8d2538-cd4b-4c75-8183-304e82cffe56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:33.135555 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8d2538-cd4b-4c75-8183-304e82cffe56 service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:33.147358 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1397/2758] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:33:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:33:33.947057 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9907ef1d-d358-4813-89ff-40846dd8092c None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:33.952950 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-9907ef1d-d358-4813-89ff-40846dd8092c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:33.968460 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9907ef1d-d358-4813-89ff-40846dd8092c None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:33.975618 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db54bbd7-9baf-4af3-a46e-69ee0ee444a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:33.976538 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db54bbd7-9baf-4af3-a46e-69ee0ee444a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:33.976811 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db54bbd7-9baf-4af3-a46e-69ee0ee444a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:33.977282 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db54bbd7-9baf-4af3-a46e-69ee0ee444a2 admin admin] PATH_INFO: `/v3/projects/9209deeb8fc84dffaf32be1451d82e7a` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:33.978791 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db54bbd7-9baf-4af3-a46e-69ee0ee444a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9209deeb8fc84dffaf32be1451d82e7a)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:33.980625 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db54bbd7-9baf-4af3-a46e-69ee0ee444a2 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:34.110267 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-db54bbd7-9baf-4af3-a46e-69ee0ee444a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9209deeb8fc84dffaf32be1451d82e7a', 'request_id': 'req-db54bbd7-9baf-4af3-a46e-69ee0ee444a2'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:34.116887 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-db54bbd7-9baf-4af3-a46e-69ee0ee444a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9209deeb8fc84dffaf32be1451d82e7a', 'request_id': 'req-db54bbd7-9baf-4af3-a46e-69ee0ee444a2'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:34.208311 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-db54bbd7-9baf-4af3-a46e-69ee0ee444a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9209deeb8fc84dffaf32be1451d82e7a', 'request_id': 'req-db54bbd7-9baf-4af3-a46e-69ee0ee444a2'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:34.222804 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-db54bbd7-9baf-4af3-a46e-69ee0ee444a2 admin admin] The token cache is being invalidate because project 9209deeb8fc84dffaf32be1451d82e7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:33:34.223096 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-db54bbd7-9baf-4af3-a46e-69ee0ee444a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:34.223812 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-db54bbd7-9baf-4af3-a46e-69ee0ee444a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:34.229626 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1362/2759] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:33:33 2022] DELETE /identity/v3/projects/9209deeb8fc84dffaf32be1451d82e7a => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:33:34.234606 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540c6fab-405c-46a2-99a5-6ff08a4b2a61 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:34.235622 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540c6fab-405c-46a2-99a5-6ff08a4b2a61 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:34.235622 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540c6fab-405c-46a2-99a5-6ff08a4b2a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:34.257198 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-540c6fab-405c-46a2-99a5-6ff08a4b2a61 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:33:34.258457 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-540c6fab-405c-46a2-99a5-6ff08a4b2a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:34.337331 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1398/2760] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:33:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:33:34.345242 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bd9ceb-e99d-4eac-96bc-8d9d2f0a9d04 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:34.345605 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bd9ceb-e99d-4eac-96bc-8d9d2f0a9d04 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:34.345990 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bd9ceb-e99d-4eac-96bc-8d9d2f0a9d04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:34.380622 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-e0bd9ceb-e99d-4eac-96bc-8d9d2f0a9d04 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:33:34.388725 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-e0bd9ceb-e99d-4eac-96bc-8d9d2f0a9d04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:34.394926 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1363/2761] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:33:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:33:34.396266 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540c6fab-405c-46a2-99a5-6ff08a4b2a61 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:34.407494 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540c6fab-405c-46a2-99a5-6ff08a4b2a61 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:34.413565 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c799409-775b-4435-96a6-c428d9b128ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:34.414330 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c799409-775b-4435-96a6-c428d9b128ff admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:34.414852 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c799409-775b-4435-96a6-c428d9b128ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:34.415428 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c799409-775b-4435-96a6-c428d9b128ff admin admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:34.416346 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c799409-775b-4435-96a6-c428d9b128ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:33:34.416834 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c799409-775b-4435-96a6-c428d9b128ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:34.419073 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c799409-775b-4435-96a6-c428d9b128ff admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:34.435355 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1399/2762] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Aug 4 21:33:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:33:34.442833 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bd9ceb-e99d-4eac-96bc-8d9d2f0a9d04 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:34.456298 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bd9ceb-e99d-4eac-96bc-8d9d2f0a9d04 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:34.468487 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e23d42-469b-4adb-ae14-326d6ba152da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:34.469287 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e23d42-469b-4adb-ae14-326d6ba152da admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:34.469706 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e23d42-469b-4adb-ae14-326d6ba152da admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:34.470103 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e23d42-469b-4adb-ae14-326d6ba152da admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:34.470964 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e23d42-469b-4adb-ae14-326d6ba152da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:34.474118 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e23d42-469b-4adb-ae14-326d6ba152da admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:34.608199 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1364/2763] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:33:34 2022] POST /identity/v3/projects => generated 383 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:33:34.613503 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c799409-775b-4435-96a6-c428d9b128ff admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:34.624572 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c799409-775b-4435-96a6-c428d9b128ff admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:34.700966 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776bf206-f974-48cc-8321-59712cf028c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:34.701673 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776bf206-f974-48cc-8321-59712cf028c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:34.706466 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776bf206-f974-48cc-8321-59712cf028c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:34.706466 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776bf206-f974-48cc-8321-59712cf028c1 admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:34.706466 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776bf206-f974-48cc-8321-59712cf028c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:34.706466 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776bf206-f974-48cc-8321-59712cf028c1 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:34.847177 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1400/2764] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:33:34 2022] POST /identity/v3/users => generated 292 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:33:34.852838 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e23d42-469b-4adb-ae14-326d6ba152da admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:34.895079 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e23d42-469b-4adb-ae14-326d6ba152da admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:34.895079 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372fc6a5-5270-4aef-a4e9-49de23a4591c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:34.895079 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372fc6a5-5270-4aef-a4e9-49de23a4591c admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:34.895079 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372fc6a5-5270-4aef-a4e9-49de23a4591c admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:34.895628 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372fc6a5-5270-4aef-a4e9-49de23a4591c admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:34.895628 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372fc6a5-5270-4aef-a4e9-49de23a4591c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:34.895628 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372fc6a5-5270-4aef-a4e9-49de23a4591c admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:34.904420 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1365/2765] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:33:34 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:33:34.908621 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776bf206-f974-48cc-8321-59712cf028c1 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:34.918940 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776bf206-f974-48cc-8321-59712cf028c1 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:34.923964 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caec41f-129c-485c-a077-8153bff60ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:34.925042 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caec41f-129c-485c-a077-8153bff60ac5 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:34.925422 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caec41f-129c-485c-a077-8153bff60ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:34.925804 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caec41f-129c-485c-a077-8153bff60ac5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:34.926651 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caec41f-129c-485c-a077-8153bff60ac5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:34.929731 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caec41f-129c-485c-a077-8153bff60ac5 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:34.931952 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:34.945572 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1401/2766] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:33:34 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:33:34.950065 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372fc6a5-5270-4aef-a4e9-49de23a4591c admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:34.963728 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372fc6a5-5270-4aef-a4e9-49de23a4591c admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:34.970704 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e130e1-0a18-45c1-8e0f-108bf101f968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:34.975908 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e130e1-0a18-45c1-8e0f-108bf101f968 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:34.976259 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e130e1-0a18-45c1-8e0f-108bf101f968 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:34.976552 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e130e1-0a18-45c1-8e0f-108bf101f968 admin admin] PATH_INFO: `/v3/projects/76ca8c859a9c4762afbcf40fb071c188/users/d1946505fcad422ba5416834ad66050d/roles/8feebc60363c4feb870718b1721e5324` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:34.995592 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:34.996604 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e130e1-0a18-45c1-8e0f-108bf101f968 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76ca8c859a9c4762afbcf40fb071c188, user_id=d1946505fcad422ba5416834ad66050d, role_id=8feebc60363c4feb870718b1721e5324)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:34.999711 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e130e1-0a18-45c1-8e0f-108bf101f968 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:35.153698 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1366/2767] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:33:34 2022] PUT /identity/v3/projects/76ca8c859a9c4762afbcf40fb071c188/users/d1946505fcad422ba5416834ad66050d/roles/8feebc60363c4feb870718b1721e5324 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:33:35.159957 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b988b4eb-0262-49ec-9bd3-a7634e23c968 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:35.160389 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b988b4eb-0262-49ec-9bd3-a7634e23c968 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:35.160795 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b988b4eb-0262-49ec-9bd3-a7634e23c968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:35.202151 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-b988b4eb-0262-49ec-9bd3-a7634e23c968 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:33:35.209051 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-b988b4eb-0262-49ec-9bd3-a7634e23c968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:35.290196 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1402/2768] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:33:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:33:35.311914 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e130e1-0a18-45c1-8e0f-108bf101f968 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:35.314537 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-a6e130e1-0a18-45c1-8e0f-108bf101f968 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:35.392334 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e130e1-0a18-45c1-8e0f-108bf101f968 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:35.402765 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa1cc7c-9cad-44d8-b983-40381225b859 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:35.403773 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa1cc7c-9cad-44d8-b983-40381225b859 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:35.403957 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa1cc7c-9cad-44d8-b983-40381225b859 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:35.404127 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa1cc7c-9cad-44d8-b983-40381225b859 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:35.419962 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa1cc7c-9cad-44d8-b983-40381225b859 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:35.421525 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa1cc7c-9cad-44d8-b983-40381225b859 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:35.433735 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1367/2769] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:33:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:33:36.919126 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c003dc-49b5-406b-9ebd-ff9f7007dd79 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:36.919943 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c003dc-49b5-406b-9ebd-ff9f7007dd79 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:36.919943 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c003dc-49b5-406b-9ebd-ff9f7007dd79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:36.952730 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-99c003dc-49b5-406b-9ebd-ff9f7007dd79 None None] MFA Rules not processed for user `d1946505fcad422ba5416834ad66050d`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:33:36.954075 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-99c003dc-49b5-406b-9ebd-ff9f7007dd79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:37.029816 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1403/2770] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:33:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:33:37.042083 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa1cc7c-9cad-44d8-b983-40381225b859 service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:37.043625 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-eaa1cc7c-9cad-44d8-b983-40381225b859 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:37.105467 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa1cc7c-9cad-44d8-b983-40381225b859 service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:37.109552 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab56afc-a7a6-4261-adb7-0b478bafe3bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e31f2728056b460fa474478eb85ca1d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:37.110221 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab56afc-a7a6-4261-adb7-0b478bafe3bd service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:37.110513 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab56afc-a7a6-4261-adb7-0b478bafe3bd service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:37.110784 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab56afc-a7a6-4261-adb7-0b478bafe3bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:37.120269 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab56afc-a7a6-4261-adb7-0b478bafe3bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:37.121893 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab56afc-a7a6-4261-adb7-0b478bafe3bd service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:37.131814 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1368/2771] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:33:37 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:33:37.754418 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c003dc-49b5-406b-9ebd-ff9f7007dd79 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:37.756058 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-99c003dc-49b5-406b-9ebd-ff9f7007dd79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:37.883869 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:37.899849 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c003dc-49b5-406b-9ebd-ff9f7007dd79 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:37.910646 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844a8148-5589-4b06-9b50-30b0c3611e8d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a2ce1df87fa47e5be301916bf3245fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:37.911460 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844a8148-5589-4b06-9b50-30b0c3611e8d service nova] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:37.911780 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844a8148-5589-4b06-9b50-30b0c3611e8d service nova] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:37.912227 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844a8148-5589-4b06-9b50-30b0c3611e8d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:37.914945 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-844a8148-5589-4b06-9b50-30b0c3611e8d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:37.990108 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844a8148-5589-4b06-9b50-30b0c3611e8d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:37.991945 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844a8148-5589-4b06-9b50-30b0c3611e8d service nova] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:38.003019 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1404/2772] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:33:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:33:38.811583 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab56afc-a7a6-4261-adb7-0b478bafe3bd service cinder] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:38.821644 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab56afc-a7a6-4261-adb7-0b478bafe3bd service cinder] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:38.825911 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d398cdf-2724-402d-b387-73dbc2461485 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e31f2728056b460fa474478eb85ca1d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:38.826554 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d398cdf-2724-402d-b387-73dbc2461485 service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:38.826813 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d398cdf-2724-402d-b387-73dbc2461485 service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:38.827083 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d398cdf-2724-402d-b387-73dbc2461485 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:38.829910 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-4d398cdf-2724-402d-b387-73dbc2461485 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:38.898829 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d398cdf-2724-402d-b387-73dbc2461485 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:38.900326 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d398cdf-2724-402d-b387-73dbc2461485 service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:38.915097 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1369/2773] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:33:38 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:33:38.996177 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f3d8c6-3fd2-4893-b335-6a4cc0eb3638 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:38.996758 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f3d8c6-3fd2-4893-b335-6a4cc0eb3638 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:38.996758 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f3d8c6-3fd2-4893-b335-6a4cc0eb3638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:39.022012 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-e2f3d8c6-3fd2-4893-b335-6a4cc0eb3638 None None] MFA Rules not processed for user `d1946505fcad422ba5416834ad66050d`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:33:39.023795 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-e2f3d8c6-3fd2-4893-b335-6a4cc0eb3638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:39.037799 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1405/2774] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:33:38 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:33:42.117420 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d398cdf-2724-402d-b387-73dbc2461485 service cinder] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:42.124990 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-4d398cdf-2724-402d-b387-73dbc2461485 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:42.131556 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d398cdf-2724-402d-b387-73dbc2461485 service cinder] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:42.135102 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5029b6b-b68e-4239-b02a-ad0ad83d4335 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:42.135751 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5029b6b-b68e-4239-b02a-ad0ad83d4335 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:42.136036 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5029b6b-b68e-4239-b02a-ad0ad83d4335 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:42.136313 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5029b6b-b68e-4239-b02a-ad0ad83d4335 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:42.138143 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-a5029b6b-b68e-4239-b02a-ad0ad83d4335 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:42.150893 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5029b6b-b68e-4239-b02a-ad0ad83d4335 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:42.152370 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5029b6b-b68e-4239-b02a-ad0ad83d4335 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:42.163053 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1370/2775] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:33:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:33:43.880312 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f3d8c6-3fd2-4893-b335-6a4cc0eb3638 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:43.882300 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-e2f3d8c6-3fd2-4893-b335-6a4cc0eb3638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:43.900985 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f3d8c6-3fd2-4893-b335-6a4cc0eb3638 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:43.905551 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe91f23f-8eb6-4138-a9a7-3a857171f350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:43.906193 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe91f23f-8eb6-4138-a9a7-3a857171f350 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:43.906473 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe91f23f-8eb6-4138-a9a7-3a857171f350 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:43.906747 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe91f23f-8eb6-4138-a9a7-3a857171f350 admin admin] PATH_INFO: `/v3/users/078b4076a64a4c87a11f86d85c815d36` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:43.908097 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe91f23f-8eb6-4138-a9a7-3a857171f350 admin admin] RBAC: Authorizing `identity:delete_user(user_id=078b4076a64a4c87a11f86d85c815d36)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:43.910131 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe91f23f-8eb6-4138-a9a7-3a857171f350 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:44.074644 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-fe91f23f-8eb6-4138-a9a7-3a857171f350 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '078b4076a64a4c87a11f86d85c815d36'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:44.088106 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-fe91f23f-8eb6-4138-a9a7-3a857171f350 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '078b4076a64a4c87a11f86d85c815d36'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:44.088805 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-fe91f23f-8eb6-4138-a9a7-3a857171f350 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '078b4076a64a4c87a11f86d85c815d36'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:44.089411 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-fe91f23f-8eb6-4138-a9a7-3a857171f350 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '078b4076a64a4c87a11f86d85c815d36'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:44.185443 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-fe91f23f-8eb6-4138-a9a7-3a857171f350 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '078b4076a64a4c87a11f86d85c815d36'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:44.207440 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1406/2776] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:33:43 2022] DELETE /identity/v3/users/078b4076a64a4c87a11f86d85c815d36 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:33:44.208883 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e87a426-691d-4854-a1f0-a2654602e951 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:44.209385 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e87a426-691d-4854-a1f0-a2654602e951 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:44.209385 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e87a426-691d-4854-a1f0-a2654602e951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:44.231733 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-4e87a426-691d-4854-a1f0-a2654602e951 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:33:44.234093 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-4e87a426-691d-4854-a1f0-a2654602e951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:44.257706 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:44.310692 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1371/2777] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:33:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:33:44.321494 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe91f23f-8eb6-4138-a9a7-3a857171f350 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:44.323886 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-fe91f23f-8eb6-4138-a9a7-3a857171f350 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:44.401259 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe91f23f-8eb6-4138-a9a7-3a857171f350 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:44.405388 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7b19ec-cbda-40ef-8756-c1405cde7eac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:44.406037 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7b19ec-cbda-40ef-8756-c1405cde7eac service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:44.406318 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7b19ec-cbda-40ef-8756-c1405cde7eac service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:44.406590 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7b19ec-cbda-40ef-8756-c1405cde7eac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:44.418530 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7b19ec-cbda-40ef-8756-c1405cde7eac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:44.420411 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7b19ec-cbda-40ef-8756-c1405cde7eac service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:44.438150 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1407/2778] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:33:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:33:44.979092 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e87a426-691d-4854-a1f0-a2654602e951 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:44.979092 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-4e87a426-691d-4854-a1f0-a2654602e951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:45.006145 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e87a426-691d-4854-a1f0-a2654602e951 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:45.015164 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a6e007-2450-4ee5-8a48-5327586a6e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:45.016191 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a6e007-2450-4ee5-8a48-5327586a6e7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:45.016391 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a6e007-2450-4ee5-8a48-5327586a6e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:45.016683 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a6e007-2450-4ee5-8a48-5327586a6e7f admin admin] PATH_INFO: `/v3/projects/612ab67992124f61934e96912f1cce3c` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:45.017956 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a6e007-2450-4ee5-8a48-5327586a6e7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=612ab67992124f61934e96912f1cce3c)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:45.020783 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a6e007-2450-4ee5-8a48-5327586a6e7f admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:45.176193 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-52a6e007-2450-4ee5-8a48-5327586a6e7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '612ab67992124f61934e96912f1cce3c', 'request_id': 'req-52a6e007-2450-4ee5-8a48-5327586a6e7f'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:45.182273 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-52a6e007-2450-4ee5-8a48-5327586a6e7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '612ab67992124f61934e96912f1cce3c', 'request_id': 'req-52a6e007-2450-4ee5-8a48-5327586a6e7f'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:45.259338 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-52a6e007-2450-4ee5-8a48-5327586a6e7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '612ab67992124f61934e96912f1cce3c', 'request_id': 'req-52a6e007-2450-4ee5-8a48-5327586a6e7f'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:45.271650 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-52a6e007-2450-4ee5-8a48-5327586a6e7f admin admin] The token cache is being invalidate because project 612ab67992124f61934e96912f1cce3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:33:45.271940 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-52a6e007-2450-4ee5-8a48-5327586a6e7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:45.272747 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-52a6e007-2450-4ee5-8a48-5327586a6e7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:45.274558 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1372/2779] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:33:44 2022] DELETE /identity/v3/projects/612ab67992124f61934e96912f1cce3c => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:33:45.279834 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7b19ec-cbda-40ef-8756-c1405cde7eac service neutron] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:45.287379 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:45.291293 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7b19ec-cbda-40ef-8756-c1405cde7eac service neutron] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:45.348334 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c39914-6cbb-4b9d-8231-434b83d33b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:45.348989 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c39914-6cbb-4b9d-8231-434b83d33b48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:45.349272 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c39914-6cbb-4b9d-8231-434b83d33b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:45.349549 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c39914-6cbb-4b9d-8231-434b83d33b48 admin admin] PATH_INFO: `/v3/users/6a70b0a2066b4f02a92e9c0a99ec2ff8` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:45.350746 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c39914-6cbb-4b9d-8231-434b83d33b48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a70b0a2066b4f02a92e9c0a99ec2ff8)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:45.352765 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c39914-6cbb-4b9d-8231-434b83d33b48 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:45.576970 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-09c39914-6cbb-4b9d-8231-434b83d33b48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a70b0a2066b4f02a92e9c0a99ec2ff8'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:45.589880 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-09c39914-6cbb-4b9d-8231-434b83d33b48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a70b0a2066b4f02a92e9c0a99ec2ff8'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:45.590663 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-09c39914-6cbb-4b9d-8231-434b83d33b48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a70b0a2066b4f02a92e9c0a99ec2ff8'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:45.591274 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-09c39914-6cbb-4b9d-8231-434b83d33b48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a70b0a2066b4f02a92e9c0a99ec2ff8'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:45.660051 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-09c39914-6cbb-4b9d-8231-434b83d33b48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a70b0a2066b4f02a92e9c0a99ec2ff8'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:45.675968 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1408/2780] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:33:45 2022] DELETE /identity/v3/users/6a70b0a2066b4f02a92e9c0a99ec2ff8 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:33:46.471675 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a6e007-2450-4ee5-8a48-5327586a6e7f admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:46.473877 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-52a6e007-2450-4ee5-8a48-5327586a6e7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:46.487703 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a6e007-2450-4ee5-8a48-5327586a6e7f admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:46.587058 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe86a2ec-6304-46b0-8852-5e9ebbeecdf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:46.589042 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe86a2ec-6304-46b0-8852-5e9ebbeecdf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:46.589568 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe86a2ec-6304-46b0-8852-5e9ebbeecdf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:46.590232 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe86a2ec-6304-46b0-8852-5e9ebbeecdf9 admin admin] PATH_INFO: `/v3/projects/69f75d920fbf47d99dfe6e49349b0637` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:46.591911 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe86a2ec-6304-46b0-8852-5e9ebbeecdf9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=69f75d920fbf47d99dfe6e49349b0637)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:46.594341 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe86a2ec-6304-46b0-8852-5e9ebbeecdf9 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:46.768379 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:46.770724 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-fe86a2ec-6304-46b0-8852-5e9ebbeecdf9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69f75d920fbf47d99dfe6e49349b0637', 'request_id': 'req-fe86a2ec-6304-46b0-8852-5e9ebbeecdf9'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:46.777788 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-fe86a2ec-6304-46b0-8852-5e9ebbeecdf9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69f75d920fbf47d99dfe6e49349b0637', 'request_id': 'req-fe86a2ec-6304-46b0-8852-5e9ebbeecdf9'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:46.858578 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-fe86a2ec-6304-46b0-8852-5e9ebbeecdf9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69f75d920fbf47d99dfe6e49349b0637', 'request_id': 'req-fe86a2ec-6304-46b0-8852-5e9ebbeecdf9'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:46.876277 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-fe86a2ec-6304-46b0-8852-5e9ebbeecdf9 admin admin] The token cache is being invalidate because project 69f75d920fbf47d99dfe6e49349b0637 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:33:46.876792 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-fe86a2ec-6304-46b0-8852-5e9ebbeecdf9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:46.878137 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-fe86a2ec-6304-46b0-8852-5e9ebbeecdf9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:46.881292 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1373/2781] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:33:46 2022] DELETE /identity/v3/projects/69f75d920fbf47d99dfe6e49349b0637 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:33:46.886096 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c39914-6cbb-4b9d-8231-434b83d33b48 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:46.890983 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-09c39914-6cbb-4b9d-8231-434b83d33b48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:46.911579 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c39914-6cbb-4b9d-8231-434b83d33b48 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:47.000068 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b46a24-bca0-49b4-9189-0cf3aa86bf09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:47.004966 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b46a24-bca0-49b4-9189-0cf3aa86bf09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:47.004966 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b46a24-bca0-49b4-9189-0cf3aa86bf09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:47.004966 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b46a24-bca0-49b4-9189-0cf3aa86bf09 admin admin] PATH_INFO: `/v3/users/d96dd5eafc7341c8b9ff6b80ca01cdd4` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:47.004966 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b46a24-bca0-49b4-9189-0cf3aa86bf09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d96dd5eafc7341c8b9ff6b80ca01cdd4)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:47.009645 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b46a24-bca0-49b4-9189-0cf3aa86bf09 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:47.145545 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:47.311172 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-68b46a24-bca0-49b4-9189-0cf3aa86bf09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd96dd5eafc7341c8b9ff6b80ca01cdd4'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:47.330458 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-68b46a24-bca0-49b4-9189-0cf3aa86bf09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd96dd5eafc7341c8b9ff6b80ca01cdd4'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:47.330458 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-68b46a24-bca0-49b4-9189-0cf3aa86bf09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd96dd5eafc7341c8b9ff6b80ca01cdd4'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:47.330458 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-68b46a24-bca0-49b4-9189-0cf3aa86bf09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd96dd5eafc7341c8b9ff6b80ca01cdd4'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:47.431821 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-68b46a24-bca0-49b4-9189-0cf3aa86bf09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd96dd5eafc7341c8b9ff6b80ca01cdd4'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:47.444755 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1409/2782] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:33:46 2022] DELETE /identity/v3/users/d96dd5eafc7341c8b9ff6b80ca01cdd4 => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:33:47.893774 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe86a2ec-6304-46b0-8852-5e9ebbeecdf9 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:47.896117 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-fe86a2ec-6304-46b0-8852-5e9ebbeecdf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:47.912319 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe86a2ec-6304-46b0-8852-5e9ebbeecdf9 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:47.974895 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32b8c02-668d-40db-9654-740eb0411bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:47.975902 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32b8c02-668d-40db-9654-740eb0411bba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:47.976355 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32b8c02-668d-40db-9654-740eb0411bba admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:47.976631 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32b8c02-668d-40db-9654-740eb0411bba admin admin] PATH_INFO: `/v3/projects/4625c6b2d0c54e90a43bbeda3c027792` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:47.977804 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32b8c02-668d-40db-9654-740eb0411bba admin admin] RBAC: Authorizing `identity:delete_project(project_id=4625c6b2d0c54e90a43bbeda3c027792)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:47.979774 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32b8c02-668d-40db-9654-740eb0411bba admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:48.156632 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c32b8c02-668d-40db-9654-740eb0411bba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4625c6b2d0c54e90a43bbeda3c027792', 'request_id': 'req-c32b8c02-668d-40db-9654-740eb0411bba'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:48.163149 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c32b8c02-668d-40db-9654-740eb0411bba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4625c6b2d0c54e90a43bbeda3c027792', 'request_id': 'req-c32b8c02-668d-40db-9654-740eb0411bba'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:48.236631 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c32b8c02-668d-40db-9654-740eb0411bba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4625c6b2d0c54e90a43bbeda3c027792', 'request_id': 'req-c32b8c02-668d-40db-9654-740eb0411bba'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:48.250043 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c32b8c02-668d-40db-9654-740eb0411bba admin admin] The token cache is being invalidate because project 4625c6b2d0c54e90a43bbeda3c027792 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:33:48.250335 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c32b8c02-668d-40db-9654-740eb0411bba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:48.251196 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c32b8c02-668d-40db-9654-740eb0411bba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:48.254522 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1374/2783] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:33:47 2022] DELETE /identity/v3/projects/4625c6b2d0c54e90a43bbeda3c027792 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:33:48.260804 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18feaa64-7ec2-47c5-a14c-a8b9ca4ce5f1 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:48.261962 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18feaa64-7ec2-47c5-a14c-a8b9ca4ce5f1 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:48.262472 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18feaa64-7ec2-47c5-a14c-a8b9ca4ce5f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:48.297945 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-18feaa64-7ec2-47c5-a14c-a8b9ca4ce5f1 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:33:48.300118 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-18feaa64-7ec2-47c5-a14c-a8b9ca4ce5f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:48.370537 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1410/2784] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:33:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:33:48.379508 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d1f3d-bfa2-43f8-9cbc-5d83315706c5 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:48.383621 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d1f3d-bfa2-43f8-9cbc-5d83315706c5 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:48.383621 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d1f3d-bfa2-43f8-9cbc-5d83315706c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:48.400818 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-182d1f3d-bfa2-43f8-9cbc-5d83315706c5 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:33:48.401960 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-182d1f3d-bfa2-43f8-9cbc-5d83315706c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:48.410502 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1375/2785] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:33:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:33:48.414424 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18feaa64-7ec2-47c5-a14c-a8b9ca4ce5f1 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:48.425313 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18feaa64-7ec2-47c5-a14c-a8b9ca4ce5f1 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:48.431193 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dca4d90-4c31-4ddd-8045-7c2e66173d0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:48.435560 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dca4d90-4c31-4ddd-8045-7c2e66173d0f admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:48.435560 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dca4d90-4c31-4ddd-8045-7c2e66173d0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:48.435560 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dca4d90-4c31-4ddd-8045-7c2e66173d0f admin admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:48.435560 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dca4d90-4c31-4ddd-8045-7c2e66173d0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:33:48.435560 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dca4d90-4c31-4ddd-8045-7c2e66173d0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:48.435560 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dca4d90-4c31-4ddd-8045-7c2e66173d0f admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:48.476331 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1411/2786] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Aug 4 21:33:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:33:48.477330 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182d1f3d-bfa2-43f8-9cbc-5d83315706c5 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:48.494534 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182d1f3d-bfa2-43f8-9cbc-5d83315706c5 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:48.507797 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f912230d-e8fe-48e7-a508-f223e6603de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:48.508498 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f912230d-e8fe-48e7-a508-f223e6603de4 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:48.508941 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f912230d-e8fe-48e7-a508-f223e6603de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:48.510211 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f912230d-e8fe-48e7-a508-f223e6603de4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:48.510211 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f912230d-e8fe-48e7-a508-f223e6603de4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:48.512101 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f912230d-e8fe-48e7-a508-f223e6603de4 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:48.595968 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1376/2787] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:33:48 2022] POST /identity/v3/projects => generated 375 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:33:48.598024 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dca4d90-4c31-4ddd-8045-7c2e66173d0f admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:48.608686 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dca4d90-4c31-4ddd-8045-7c2e66173d0f admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:48.687043 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc21867-ab9b-4769-9dd3-0039907b6c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:48.687743 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc21867-ab9b-4769-9dd3-0039907b6c42 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:48.688035 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc21867-ab9b-4769-9dd3-0039907b6c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:48.688296 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc21867-ab9b-4769-9dd3-0039907b6c42 admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:48.688990 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc21867-ab9b-4769-9dd3-0039907b6c42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:48.690935 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc21867-ab9b-4769-9dd3-0039907b6c42 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:48.781588 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1412/2788] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:33:48 2022] POST /identity/v3/users => generated 288 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:33:48.786472 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f912230d-e8fe-48e7-a508-f223e6603de4 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:48.797883 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f912230d-e8fe-48e7-a508-f223e6603de4 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:48.803189 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239c8394-c1ca-4313-8f34-398f8b70c3d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:48.803944 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239c8394-c1ca-4313-8f34-398f8b70c3d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:48.809644 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239c8394-c1ca-4313-8f34-398f8b70c3d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:48.809644 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239c8394-c1ca-4313-8f34-398f8b70c3d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:48.809644 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239c8394-c1ca-4313-8f34-398f8b70c3d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:48.809644 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239c8394-c1ca-4313-8f34-398f8b70c3d1 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:48.819030 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1377/2789] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:33:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:33:48.824226 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc21867-ab9b-4769-9dd3-0039907b6c42 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:48.837499 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc21867-ab9b-4769-9dd3-0039907b6c42 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:48.846799 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06c263f-9308-4835-9f09-f94cf116a264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:48.848242 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c263f-9308-4835-9f09-f94cf116a264 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:48.848671 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c263f-9308-4835-9f09-f94cf116a264 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:48.849023 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c263f-9308-4835-9f09-f94cf116a264 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:48.849894 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06c263f-9308-4835-9f09-f94cf116a264 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:48.853167 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06c263f-9308-4835-9f09-f94cf116a264 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:48.866428 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1413/2790] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:33:48 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:33:48.870753 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239c8394-c1ca-4313-8f34-398f8b70c3d1 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:48.881998 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239c8394-c1ca-4313-8f34-398f8b70c3d1 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:48.886876 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1418b5-d37e-46a4-859d-e708908299f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:48.887666 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1418b5-d37e-46a4-859d-e708908299f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:48.887948 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1418b5-d37e-46a4-859d-e708908299f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:48.888223 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1418b5-d37e-46a4-859d-e708908299f2 admin admin] PATH_INFO: `/v3/projects/fbf398b29b90407db08b028f5c4f8160/users/d422a2f2b6a04116805a008ccccc8e7d/roles/8feebc60363c4feb870718b1721e5324` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:48.903574 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1418b5-d37e-46a4-859d-e708908299f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbf398b29b90407db08b028f5c4f8160, user_id=d422a2f2b6a04116805a008ccccc8e7d, role_id=8feebc60363c4feb870718b1721e5324)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:48.906347 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1418b5-d37e-46a4-859d-e708908299f2 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:49.001417 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1378/2791] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:33:48 2022] PUT /identity/v3/projects/fbf398b29b90407db08b028f5c4f8160/users/d422a2f2b6a04116805a008ccccc8e7d/roles/8feebc60363c4feb870718b1721e5324 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:33:49.013986 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbbbccb-c5fb-4ed1-b16b-f0853bb8e666 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:49.014274 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbbbccb-c5fb-4ed1-b16b-f0853bb8e666 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:49.014651 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbbbccb-c5fb-4ed1-b16b-f0853bb8e666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:49.035370 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:49.045054 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-7bbbbccb-c5fb-4ed1-b16b-f0853bb8e666 None None] MFA Rules not processed for user `d422a2f2b6a04116805a008ccccc8e7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:33:49.047103 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-7bbbbccb-c5fb-4ed1-b16b-f0853bb8e666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:49.121060 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1414/2792] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:33:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:33:49.122358 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1418b5-d37e-46a4-859d-e708908299f2 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:49.127037 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:49.189388 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1418b5-d37e-46a4-859d-e708908299f2 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:49.194169 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30208528-146a-4bb1-b90c-62cf32b38ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:49.194786 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30208528-146a-4bb1-b90c-62cf32b38ccd admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:49.195072 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30208528-146a-4bb1-b90c-62cf32b38ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:49.195370 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30208528-146a-4bb1-b90c-62cf32b38ccd admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:49.196032 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30208528-146a-4bb1-b90c-62cf32b38ccd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:49.198380 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30208528-146a-4bb1-b90c-62cf32b38ccd admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:49.288327 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1379/2793] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:33:49 2022] POST /identity/v3/projects => generated 375 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:33:49.289540 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbbbccb-c5fb-4ed1-b16b-f0853bb8e666 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:49.301524 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbbbccb-c5fb-4ed1-b16b-f0853bb8e666 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:49.364614 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24e347c-7f86-4037-b712-99e7cf7696b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:49.365255 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24e347c-7f86-4037-b712-99e7cf7696b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:49.365570 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24e347c-7f86-4037-b712-99e7cf7696b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:49.365802 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24e347c-7f86-4037-b712-99e7cf7696b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:49.366511 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24e347c-7f86-4037-b712-99e7cf7696b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:49.368523 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24e347c-7f86-4037-b712-99e7cf7696b7 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:49.500373 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1415/2794] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:33:49 2022] POST /identity/v3/users => generated 294 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:33:49.504954 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30208528-146a-4bb1-b90c-62cf32b38ccd admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:49.516725 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30208528-146a-4bb1-b90c-62cf32b38ccd admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:49.521495 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36bfaf0-e10c-4f73-92a8-f573577ed741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:49.522127 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36bfaf0-e10c-4f73-92a8-f573577ed741 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:49.522412 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36bfaf0-e10c-4f73-92a8-f573577ed741 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:49.522860 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36bfaf0-e10c-4f73-92a8-f573577ed741 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:49.523606 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36bfaf0-e10c-4f73-92a8-f573577ed741 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:49.525245 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36bfaf0-e10c-4f73-92a8-f573577ed741 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:49.545216 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1380/2795] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:33:49 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:33:49.546401 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24e347c-7f86-4037-b712-99e7cf7696b7 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:49.559983 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24e347c-7f86-4037-b712-99e7cf7696b7 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:49.566520 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8cd796-5227-4d92-849d-445b76dffb0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:49.566520 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8cd796-5227-4d92-849d-445b76dffb0d admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:49.566957 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8cd796-5227-4d92-849d-445b76dffb0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:49.567054 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8cd796-5227-4d92-849d-445b76dffb0d admin admin] PATH_INFO: `/v3/projects/3428760e9c794d59a9b257b7b8f7aade/users/89884e9165dd49518a110b3c003e85f4/roles/a432dd4735944ecfab2fa5ce1705d356` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:49.582834 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8cd796-5227-4d92-849d-445b76dffb0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3428760e9c794d59a9b257b7b8f7aade, user_id=89884e9165dd49518a110b3c003e85f4, role_id=a432dd4735944ecfab2fa5ce1705d356)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:49.585525 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8cd796-5227-4d92-849d-445b76dffb0d admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:49.699629 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1416/2796] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:33:49 2022] PUT /identity/v3/projects/3428760e9c794d59a9b257b7b8f7aade/users/89884e9165dd49518a110b3c003e85f4/roles/a432dd4735944ecfab2fa5ce1705d356 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:33:49.700856 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36bfaf0-e10c-4f73-92a8-f573577ed741 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:49.711944 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36bfaf0-e10c-4f73-92a8-f573577ed741 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:49.849591 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8f1f82-c5c5-4cb6-bd56-09a7d446b3b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:49.850842 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8f1f82-c5c5-4cb6-bd56-09a7d446b3b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:49.851639 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8f1f82-c5c5-4cb6-bd56-09a7d446b3b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:49.851639 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8f1f82-c5c5-4cb6-bd56-09a7d446b3b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:49.852432 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8f1f82-c5c5-4cb6-bd56-09a7d446b3b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:49.854355 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8f1f82-c5c5-4cb6-bd56-09a7d446b3b2 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:49.867367 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1381/2797] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:33:49 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:33:49.871905 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8cd796-5227-4d92-849d-445b76dffb0d admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:49.890942 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8cd796-5227-4d92-849d-445b76dffb0d admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:49.897250 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8f2845-c833-43bc-8bcc-7e370fcd732d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:49.898023 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f2845-c833-43bc-8bcc-7e370fcd732d admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:49.898377 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f2845-c833-43bc-8bcc-7e370fcd732d admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:49.898771 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f2845-c833-43bc-8bcc-7e370fcd732d admin admin] PATH_INFO: `/v3/projects/3428760e9c794d59a9b257b7b8f7aade/users/89884e9165dd49518a110b3c003e85f4/roles/a432dd4735944ecfab2fa5ce1705d356` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:49.902061 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8f2845-c833-43bc-8bcc-7e370fcd732d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3428760e9c794d59a9b257b7b8f7aade, user_id=89884e9165dd49518a110b3c003e85f4, role_id=a432dd4735944ecfab2fa5ce1705d356)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:49.906530 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8f2845-c833-43bc-8bcc-7e370fcd732d admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:49.925495 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1417/2798] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:33:49 2022] PUT /identity/v3/projects/3428760e9c794d59a9b257b7b8f7aade/users/89884e9165dd49518a110b3c003e85f4/roles/a432dd4735944ecfab2fa5ce1705d356 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:33:49.938307 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24daf9df-7d85-45fd-8bfd-42e1f20c1886 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:49.938858 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24daf9df-7d85-45fd-8bfd-42e1f20c1886 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:49.939274 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24daf9df-7d85-45fd-8bfd-42e1f20c1886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:49.979545 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-24daf9df-7d85-45fd-8bfd-42e1f20c1886 None None] MFA Rules not processed for user `89884e9165dd49518a110b3c003e85f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:33:49.980916 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-24daf9df-7d85-45fd-8bfd-42e1f20c1886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:50.079267 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:50.085014 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1382/2799] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:33:49 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:33:50.096835 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8f2845-c833-43bc-8bcc-7e370fcd732d admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:50.098380 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-3f8f2845-c833-43bc-8bcc-7e370fcd732d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:50.179720 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8f2845-c833-43bc-8bcc-7e370fcd732d admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:50.184495 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721f8d02-c718-4b8e-bf3f-408a26132fa6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e31f2728056b460fa474478eb85ca1d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:50.185108 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721f8d02-c718-4b8e-bf3f-408a26132fa6 service cinder] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:50.185463 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721f8d02-c718-4b8e-bf3f-408a26132fa6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:50.185655 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721f8d02-c718-4b8e-bf3f-408a26132fa6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:50.190238 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:50.249263 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721f8d02-c718-4b8e-bf3f-408a26132fa6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:50.251456 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721f8d02-c718-4b8e-bf3f-408a26132fa6 service cinder] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:50.263773 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1418/2800] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:33:50 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:33:52.056871 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24daf9df-7d85-45fd-8bfd-42e1f20c1886 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:52.064260 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-24daf9df-7d85-45fd-8bfd-42e1f20c1886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:52.152077 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24daf9df-7d85-45fd-8bfd-42e1f20c1886 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:52.157206 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e33942-9a99-4674-b1f8-07d8fc43497a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:52.158000 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e33942-9a99-4674-b1f8-07d8fc43497a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:52.158353 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e33942-9a99-4674-b1f8-07d8fc43497a admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:52.158697 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e33942-9a99-4674-b1f8-07d8fc43497a admin admin] PATH_INFO: `/v3/users/f480dbdc7237420680584ce4866b36e9` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:52.161048 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e33942-9a99-4674-b1f8-07d8fc43497a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f480dbdc7237420680584ce4866b36e9)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:52.176996 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e33942-9a99-4674-b1f8-07d8fc43497a admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:52.300946 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721f8d02-c718-4b8e-bf3f-408a26132fa6 service cinder] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:52.302747 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-721f8d02-c718-4b8e-bf3f-408a26132fa6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:52.318730 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721f8d02-c718-4b8e-bf3f-408a26132fa6 service cinder] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:52.327690 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d946c9ac-a948-4b1c-b587-f4dbbb065ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:52.328564 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d946c9ac-a948-4b1c-b587-f4dbbb065ea1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:52.328856 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d946c9ac-a948-4b1c-b587-f4dbbb065ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:52.329133 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d946c9ac-a948-4b1c-b587-f4dbbb065ea1 admin admin] PATH_INFO: `/v3/users/c0645e94946f4d44b03b84a4665d2ac4` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:52.333323 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d946c9ac-a948-4b1c-b587-f4dbbb065ea1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0645e94946f4d44b03b84a4665d2ac4)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:52.333976 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d946c9ac-a948-4b1c-b587-f4dbbb065ea1 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:52.444397 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-28e33942-9a99-4674-b1f8-07d8fc43497a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f480dbdc7237420680584ce4866b36e9'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:52.465220 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-28e33942-9a99-4674-b1f8-07d8fc43497a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f480dbdc7237420680584ce4866b36e9'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:52.465220 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-28e33942-9a99-4674-b1f8-07d8fc43497a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f480dbdc7237420680584ce4866b36e9'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:52.469992 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-28e33942-9a99-4674-b1f8-07d8fc43497a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f480dbdc7237420680584ce4866b36e9'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:52.470767 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-28e33942-9a99-4674-b1f8-07d8fc43497a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f480dbdc7237420680584ce4866b36e9'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:52.579799 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1383/2801] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:33:52 2022] DELETE /identity/v3/users/f480dbdc7237420680584ce4866b36e9 => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:33:52.592621 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-d946c9ac-a948-4b1c-b587-f4dbbb065ea1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0645e94946f4d44b03b84a4665d2ac4'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:52.606425 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-d946c9ac-a948-4b1c-b587-f4dbbb065ea1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0645e94946f4d44b03b84a4665d2ac4'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:52.607230 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-d946c9ac-a948-4b1c-b587-f4dbbb065ea1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0645e94946f4d44b03b84a4665d2ac4'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:52.607906 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-d946c9ac-a948-4b1c-b587-f4dbbb065ea1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0645e94946f4d44b03b84a4665d2ac4'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:52.682257 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-d946c9ac-a948-4b1c-b587-f4dbbb065ea1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0645e94946f4d44b03b84a4665d2ac4'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:52.696537 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1419/2802] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:33:52 2022] DELETE /identity/v3/users/c0645e94946f4d44b03b84a4665d2ac4 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:33:52.701650 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2a981e-e88f-4f9b-b68b-a358619f0542 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:52.701942 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2a981e-e88f-4f9b-b68b-a358619f0542 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:52.702225 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2a981e-e88f-4f9b-b68b-a358619f0542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:52.723303 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-ac2a981e-e88f-4f9b-b68b-a358619f0542 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:33:52.725900 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-ac2a981e-e88f-4f9b-b68b-a358619f0542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:52.787249 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1384/2803] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:33:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:33:52.797163 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d946c9ac-a948-4b1c-b587-f4dbbb065ea1 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:52.798586 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-d946c9ac-a948-4b1c-b587-f4dbbb065ea1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:52.868347 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d946c9ac-a948-4b1c-b587-f4dbbb065ea1 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:52.872333 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ea940f-6077-4eba-ae20-76e6faa64157 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:52.903087 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ea940f-6077-4eba-ae20-76e6faa64157 service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:52.903087 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ea940f-6077-4eba-ae20-76e6faa64157 service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:52.903087 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ea940f-6077-4eba-ae20-76e6faa64157 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:52.903087 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ea940f-6077-4eba-ae20-76e6faa64157 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:52.903087 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ea940f-6077-4eba-ae20-76e6faa64157 service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:52.903087 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1420/2804] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:33:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:33:53.003792 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2a981e-e88f-4f9b-b68b-a358619f0542 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:53.005221 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-ac2a981e-e88f-4f9b-b68b-a358619f0542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:53.018613 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2a981e-e88f-4f9b-b68b-a358619f0542 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:53.023873 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0646c825-cc43-4495-8092-6df00a0e5572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:53.024511 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0646c825-cc43-4495-8092-6df00a0e5572 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:53.024822 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0646c825-cc43-4495-8092-6df00a0e5572 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:53.025643 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0646c825-cc43-4495-8092-6df00a0e5572 admin admin] PATH_INFO: `/v3/projects/f1b4cfcf22614a95ad77db93922449cd` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:53.026330 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0646c825-cc43-4495-8092-6df00a0e5572 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1b4cfcf22614a95ad77db93922449cd)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:53.029602 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0646c825-cc43-4495-8092-6df00a0e5572 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:53.042171 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:53.166946 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-0646c825-cc43-4495-8092-6df00a0e5572 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1b4cfcf22614a95ad77db93922449cd', 'request_id': 'req-0646c825-cc43-4495-8092-6df00a0e5572'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:53.174150 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-0646c825-cc43-4495-8092-6df00a0e5572 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1b4cfcf22614a95ad77db93922449cd', 'request_id': 'req-0646c825-cc43-4495-8092-6df00a0e5572'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:53.254394 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-0646c825-cc43-4495-8092-6df00a0e5572 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1b4cfcf22614a95ad77db93922449cd', 'request_id': 'req-0646c825-cc43-4495-8092-6df00a0e5572'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:53.266952 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-0646c825-cc43-4495-8092-6df00a0e5572 admin admin] The token cache is being invalidate because project f1b4cfcf22614a95ad77db93922449cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:33:53.267177 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-0646c825-cc43-4495-8092-6df00a0e5572 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:53.267874 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-0646c825-cc43-4495-8092-6df00a0e5572 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:53.269456 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1385/2805] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:33:53 2022] DELETE /identity/v3/projects/f1b4cfcf22614a95ad77db93922449cd => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:33:53.273677 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ea940f-6077-4eba-ae20-76e6faa64157 service neutron] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:53.284820 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ea940f-6077-4eba-ae20-76e6faa64157 service neutron] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:53.303293 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:53.348114 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0646c825-cc43-4495-8092-6df00a0e5572 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:53.350387 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-0646c825-cc43-4495-8092-6df00a0e5572 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:53.362499 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83783603-8830-4547-a7a3-a0984045d4d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:53.363578 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83783603-8830-4547-a7a3-a0984045d4d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:53.364046 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83783603-8830-4547-a7a3-a0984045d4d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:53.364506 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83783603-8830-4547-a7a3-a0984045d4d9 admin admin] PATH_INFO: `/v3/users/100ea03c7bfd4d41b799301f9ea5d655` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:53.366534 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83783603-8830-4547-a7a3-a0984045d4d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=100ea03c7bfd4d41b799301f9ea5d655)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:53.370157 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83783603-8830-4547-a7a3-a0984045d4d9 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:53.443175 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0646c825-cc43-4495-8092-6df00a0e5572 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:53.448396 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b869a302-65ce-43ab-9a99-8aaa21b6bbfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:53.449038 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b869a302-65ce-43ab-9a99-8aaa21b6bbfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:53.449322 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b869a302-65ce-43ab-9a99-8aaa21b6bbfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:53.449598 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b869a302-65ce-43ab-9a99-8aaa21b6bbfb admin admin] PATH_INFO: `/v3/projects/b43abc1ee5a543ccb8930f235403a47e` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:53.450747 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b869a302-65ce-43ab-9a99-8aaa21b6bbfb admin admin] RBAC: Authorizing `identity:delete_project(project_id=b43abc1ee5a543ccb8930f235403a47e)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:53.452768 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b869a302-65ce-43ab-9a99-8aaa21b6bbfb admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:53.594907 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-b869a302-65ce-43ab-9a99-8aaa21b6bbfb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b43abc1ee5a543ccb8930f235403a47e', 'request_id': 'req-b869a302-65ce-43ab-9a99-8aaa21b6bbfb'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:53.597644 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-83783603-8830-4547-a7a3-a0984045d4d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '100ea03c7bfd4d41b799301f9ea5d655'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:53.602066 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-b869a302-65ce-43ab-9a99-8aaa21b6bbfb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b43abc1ee5a543ccb8930f235403a47e', 'request_id': 'req-b869a302-65ce-43ab-9a99-8aaa21b6bbfb'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:53.610527 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-83783603-8830-4547-a7a3-a0984045d4d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '100ea03c7bfd4d41b799301f9ea5d655'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:53.611263 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-83783603-8830-4547-a7a3-a0984045d4d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '100ea03c7bfd4d41b799301f9ea5d655'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:53.611914 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-83783603-8830-4547-a7a3-a0984045d4d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '100ea03c7bfd4d41b799301f9ea5d655'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:53.696448 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-b869a302-65ce-43ab-9a99-8aaa21b6bbfb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b43abc1ee5a543ccb8930f235403a47e', 'request_id': 'req-b869a302-65ce-43ab-9a99-8aaa21b6bbfb'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:53.711119 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-b869a302-65ce-43ab-9a99-8aaa21b6bbfb admin admin] The token cache is being invalidate because project b43abc1ee5a543ccb8930f235403a47e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:33:53.711498 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-b869a302-65ce-43ab-9a99-8aaa21b6bbfb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:53.712205 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-b869a302-65ce-43ab-9a99-8aaa21b6bbfb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:53.713959 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1386/2806] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:33:53 2022] DELETE /identity/v3/projects/b43abc1ee5a543ccb8930f235403a47e => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:33:53.714722 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b869a302-65ce-43ab-9a99-8aaa21b6bbfb admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:53.716395 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-b869a302-65ce-43ab-9a99-8aaa21b6bbfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:53.784843 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b869a302-65ce-43ab-9a99-8aaa21b6bbfb admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:53.786460 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-83783603-8830-4547-a7a3-a0984045d4d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '100ea03c7bfd4d41b799301f9ea5d655'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:53.789737 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae83d856-ea43-43f8-8841-0a996bee02a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:53.790371 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae83d856-ea43-43f8-8841-0a996bee02a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:53.790700 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae83d856-ea43-43f8-8841-0a996bee02a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:53.790977 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae83d856-ea43-43f8-8841-0a996bee02a0 admin admin] PATH_INFO: `/v3/users/d422a2f2b6a04116805a008ccccc8e7d` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:53.792942 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae83d856-ea43-43f8-8841-0a996bee02a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d422a2f2b6a04116805a008ccccc8e7d)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:53.795416 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae83d856-ea43-43f8-8841-0a996bee02a0 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:53.799614 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1421/2807] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:33:53 2022] DELETE /identity/v3/users/100ea03c7bfd4d41b799301f9ea5d655 => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:33:53.801129 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2342deb3-83f1-40d8-8f14-1a84205dff24 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:53.801279 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2342deb3-83f1-40d8-8f14-1a84205dff24 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:53.801600 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2342deb3-83f1-40d8-8f14-1a84205dff24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:53.825013 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-2342deb3-83f1-40d8-8f14-1a84205dff24 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:33:53.826249 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-2342deb3-83f1-40d8-8f14-1a84205dff24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:53.893521 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1422/2808] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:33:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:33:53.902996 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9824f88-706d-45e3-8c12-a369b22cb1fa None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:53.904146 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9824f88-706d-45e3-8c12-a369b22cb1fa None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:53.904146 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9824f88-706d-45e3-8c12-a369b22cb1fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:53.926237 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-e9824f88-706d-45e3-8c12-a369b22cb1fa None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:33:53.927504 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-e9824f88-706d-45e3-8c12-a369b22cb1fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:53.936605 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1423/2809] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:33:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:33:53.941954 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9824f88-706d-45e3-8c12-a369b22cb1fa None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:53.951490 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9824f88-706d-45e3-8c12-a369b22cb1fa None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:53.956349 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0aacbb-d737-4d00-9224-d1118a1a31af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:53.957018 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0aacbb-d737-4d00-9224-d1118a1a31af admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:53.957303 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0aacbb-d737-4d00-9224-d1118a1a31af admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:53.957588 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0aacbb-d737-4d00-9224-d1118a1a31af admin admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:53.958329 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0aacbb-d737-4d00-9224-d1118a1a31af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:33:53.958639 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0aacbb-d737-4d00-9224-d1118a1a31af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:53.960358 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0aacbb-d737-4d00-9224-d1118a1a31af admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:53.974091 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1424/2810] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Aug 4 21:33:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:33:53.978616 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0aacbb-d737-4d00-9224-d1118a1a31af admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:53.979360 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ae83d856-ea43-43f8-8841-0a996bee02a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd422a2f2b6a04116805a008ccccc8e7d'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:53.994840 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0aacbb-d737-4d00-9224-d1118a1a31af admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:54.000337 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ae83d856-ea43-43f8-8841-0a996bee02a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd422a2f2b6a04116805a008ccccc8e7d'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:54.000337 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ae83d856-ea43-43f8-8841-0a996bee02a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd422a2f2b6a04116805a008ccccc8e7d'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:54.000636 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ba5fbf-f049-405b-8fb4-c3d581d04897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:54.001336 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ba5fbf-f049-405b-8fb4-c3d581d04897 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:54.001683 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ba5fbf-f049-405b-8fb4-c3d581d04897 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:54.002239 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ba5fbf-f049-405b-8fb4-c3d581d04897 admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:54.003032 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ba5fbf-f049-405b-8fb4-c3d581d04897 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:54.005748 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ba5fbf-f049-405b-8fb4-c3d581d04897 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:54.006528 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ae83d856-ea43-43f8-8841-0a996bee02a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd422a2f2b6a04116805a008ccccc8e7d'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:54.007438 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ae83d856-ea43-43f8-8841-0a996bee02a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd422a2f2b6a04116805a008ccccc8e7d'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:54.079876 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1387/2811] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:33:53 2022] DELETE /identity/v3/users/d422a2f2b6a04116805a008ccccc8e7d => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:33:54.079876 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae83d856-ea43-43f8-8841-0a996bee02a0 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:54.081061 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-ae83d856-ea43-43f8-8841-0a996bee02a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:54.130010 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1425/2812] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:33:53 2022] POST /identity/v3/projects => generated 385 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:33:54.134418 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bd83ba-4d83-49de-b452-9c0ed5075765 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:54.134709 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bd83ba-4d83-49de-b452-9c0ed5075765 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:54.134985 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bd83ba-4d83-49de-b452-9c0ed5075765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:54.154572 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae83d856-ea43-43f8-8841-0a996bee02a0 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:54.154928 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-75bd83ba-4d83-49de-b452-9c0ed5075765 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:33:54.156343 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-75bd83ba-4d83-49de-b452-9c0ed5075765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:54.212997 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7174c0e7-f21c-4c89-963e-c67aba68de2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:54.213683 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7174c0e7-f21c-4c89-963e-c67aba68de2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:54.213977 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7174c0e7-f21c-4c89-963e-c67aba68de2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:54.214256 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7174c0e7-f21c-4c89-963e-c67aba68de2e admin admin] PATH_INFO: `/v3/projects/ee3c311493db4aa987ec4ec5cc51547e` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:54.215469 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7174c0e7-f21c-4c89-963e-c67aba68de2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee3c311493db4aa987ec4ec5cc51547e)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:54.217443 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7174c0e7-f21c-4c89-963e-c67aba68de2e admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:54.219251 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1426/2813] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:33:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:33:54.220083 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bd83ba-4d83-49de-b452-9c0ed5075765 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:54.227130 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-75bd83ba-4d83-49de-b452-9c0ed5075765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:54.247008 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bd83ba-4d83-49de-b452-9c0ed5075765 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:54.251750 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf154423-6939-4eaf-b811-744484700d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:54.252365 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf154423-6939-4eaf-b811-744484700d7b admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:54.252649 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf154423-6939-4eaf-b811-744484700d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:54.252918 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf154423-6939-4eaf-b811-744484700d7b admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:54.253606 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf154423-6939-4eaf-b811-744484700d7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:54.255587 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf154423-6939-4eaf-b811-744484700d7b admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:54.334224 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-7174c0e7-f21c-4c89-963e-c67aba68de2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee3c311493db4aa987ec4ec5cc51547e', 'request_id': 'req-7174c0e7-f21c-4c89-963e-c67aba68de2e'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:54.340291 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-7174c0e7-f21c-4c89-963e-c67aba68de2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee3c311493db4aa987ec4ec5cc51547e', 'request_id': 'req-7174c0e7-f21c-4c89-963e-c67aba68de2e'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:54.358670 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1427/2814] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:33:54 2022] POST /identity/v3/users => generated 293 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:33:54.358670 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf154423-6939-4eaf-b811-744484700d7b admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:54.358670 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-cf154423-6939-4eaf-b811-744484700d7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:54.418545 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf154423-6939-4eaf-b811-744484700d7b admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:54.437198 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-7174c0e7-f21c-4c89-963e-c67aba68de2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee3c311493db4aa987ec4ec5cc51547e', 'request_id': 'req-7174c0e7-f21c-4c89-963e-c67aba68de2e'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:54.451023 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-7174c0e7-f21c-4c89-963e-c67aba68de2e admin admin] The token cache is being invalidate because project ee3c311493db4aa987ec4ec5cc51547e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:33:54.451331 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-7174c0e7-f21c-4c89-963e-c67aba68de2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:54.452018 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-7174c0e7-f21c-4c89-963e-c67aba68de2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:54.453771 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1388/2815] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:33:54 2022] DELETE /identity/v3/projects/ee3c311493db4aa987ec4ec5cc51547e => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:33:54.454350 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7174c0e7-f21c-4c89-963e-c67aba68de2e admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:54.455827 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-7174c0e7-f21c-4c89-963e-c67aba68de2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:54.475851 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2fe27c-c9b4-4393-a8de-7c7e6667f97f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:54.476513 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2fe27c-c9b4-4393-a8de-7c7e6667f97f service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:54.478076 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2fe27c-c9b4-4393-a8de-7c7e6667f97f service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:54.478076 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2fe27c-c9b4-4393-a8de-7c7e6667f97f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:54.479074 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-bc2fe27c-c9b4-4393-a8de-7c7e6667f97f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:54.528915 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7174c0e7-f21c-4c89-963e-c67aba68de2e admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:54.531683 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:54.537696 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba73bfc-a009-4153-8c49-0489ac9df49d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:54.538447 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba73bfc-a009-4153-8c49-0489ac9df49d admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:54.538644 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba73bfc-a009-4153-8c49-0489ac9df49d admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:54.543958 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba73bfc-a009-4153-8c49-0489ac9df49d admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:54.543958 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba73bfc-a009-4153-8c49-0489ac9df49d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:54.543958 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba73bfc-a009-4153-8c49-0489ac9df49d admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:54.552604 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2fe27c-c9b4-4393-a8de-7c7e6667f97f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:54.556866 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1389/2816] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:33:54 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:33:54.557073 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2fe27c-c9b4-4393-a8de-7c7e6667f97f service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:54.557654 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba73bfc-a009-4153-8c49-0489ac9df49d admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:54.565548 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1428/2817] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:33:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:33:54.569779 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba73bfc-a009-4153-8c49-0489ac9df49d admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:54.576667 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54536fd2-241c-4a50-a5dd-a820e8a23a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:54.579911 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54536fd2-241c-4a50-a5dd-a820e8a23a3e admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:54.579911 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54536fd2-241c-4a50-a5dd-a820e8a23a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:54.579911 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54536fd2-241c-4a50-a5dd-a820e8a23a3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:54.579911 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54536fd2-241c-4a50-a5dd-a820e8a23a3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:54.582161 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54536fd2-241c-4a50-a5dd-a820e8a23a3e admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:54.595572 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1390/2818] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:33:54 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:33:54.599668 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2fe27c-c9b4-4393-a8de-7c7e6667f97f service neutron] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:54.606529 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:54.612384 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2fe27c-c9b4-4393-a8de-7c7e6667f97f service neutron] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:54.618555 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7f49e5-f2f2-42af-924d-5dd87a616e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:54.619365 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7f49e5-f2f2-42af-924d-5dd87a616e38 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:54.619662 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7f49e5-f2f2-42af-924d-5dd87a616e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:54.620260 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7f49e5-f2f2-42af-924d-5dd87a616e38 admin admin] PATH_INFO: `/v3/projects/a7664cec57944fccbca1522095567ab8/users/2fc55eab88954503807d531cadd08543/roles/8feebc60363c4feb870718b1721e5324` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:54.641992 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7f49e5-f2f2-42af-924d-5dd87a616e38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7664cec57944fccbca1522095567ab8, user_id=2fc55eab88954503807d531cadd08543, role_id=8feebc60363c4feb870718b1721e5324)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:54.648588 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7f49e5-f2f2-42af-924d-5dd87a616e38 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:54.732719 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1429/2819] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:33:54 2022] PUT /identity/v3/projects/a7664cec57944fccbca1522095567ab8/users/2fc55eab88954503807d531cadd08543/roles/8feebc60363c4feb870718b1721e5324 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:33:54.738870 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd2d99f-fd97-428d-98c2-ee2e3efd320e None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:54.739159 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd2d99f-fd97-428d-98c2-ee2e3efd320e None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:54.739487 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd2d99f-fd97-428d-98c2-ee2e3efd320e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:54.758595 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-1bd2d99f-fd97-428d-98c2-ee2e3efd320e None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:33:54.759912 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-1bd2d99f-fd97-428d-98c2-ee2e3efd320e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:54.823692 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1391/2820] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:33:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:33:54.837098 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7f49e5-f2f2-42af-924d-5dd87a616e38 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:54.839634 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-fc7f49e5-f2f2-42af-924d-5dd87a616e38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:54.903258 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7f49e5-f2f2-42af-924d-5dd87a616e38 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:54.906972 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33221aa2-c2cd-4451-9702-6197e8d65e68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:54.908109 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33221aa2-c2cd-4451-9702-6197e8d65e68 service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:54.917612 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33221aa2-c2cd-4451-9702-6197e8d65e68 service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:54.917612 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33221aa2-c2cd-4451-9702-6197e8d65e68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:54.919137 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33221aa2-c2cd-4451-9702-6197e8d65e68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:54.920664 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33221aa2-c2cd-4451-9702-6197e8d65e68 service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:54.930963 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1430/2821] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:33:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:33:55.072626 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd2d99f-fd97-428d-98c2-ee2e3efd320e None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:55.074548 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-1bd2d99f-fd97-428d-98c2-ee2e3efd320e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:55.089202 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd2d99f-fd97-428d-98c2-ee2e3efd320e None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:55.110786 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b51f4d-032b-4598-b192-d5d702e8b677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:55.111762 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b51f4d-032b-4598-b192-d5d702e8b677 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:55.112044 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b51f4d-032b-4598-b192-d5d702e8b677 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:55.112322 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b51f4d-032b-4598-b192-d5d702e8b677 admin admin] PATH_INFO: `/v3/projects/fbf398b29b90407db08b028f5c4f8160` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:55.113655 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b51f4d-032b-4598-b192-d5d702e8b677 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbf398b29b90407db08b028f5c4f8160)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:55.120026 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b51f4d-032b-4598-b192-d5d702e8b677 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:55.245828 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c1b51f4d-032b-4598-b192-d5d702e8b677 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbf398b29b90407db08b028f5c4f8160', 'request_id': 'req-c1b51f4d-032b-4598-b192-d5d702e8b677'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:55.252797 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c1b51f4d-032b-4598-b192-d5d702e8b677 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbf398b29b90407db08b028f5c4f8160', 'request_id': 'req-c1b51f4d-032b-4598-b192-d5d702e8b677'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:55.344840 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c1b51f4d-032b-4598-b192-d5d702e8b677 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbf398b29b90407db08b028f5c4f8160', 'request_id': 'req-c1b51f4d-032b-4598-b192-d5d702e8b677'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:55.357936 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c1b51f4d-032b-4598-b192-d5d702e8b677 admin admin] The token cache is being invalidate because project fbf398b29b90407db08b028f5c4f8160 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:33:55.358141 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c1b51f4d-032b-4598-b192-d5d702e8b677 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:55.358799 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c1b51f4d-032b-4598-b192-d5d702e8b677 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:55.360568 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1392/2822] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:33:55 2022] DELETE /identity/v3/projects/fbf398b29b90407db08b028f5c4f8160 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:33:55.365260 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33221aa2-c2cd-4451-9702-6197e8d65e68 service neutron] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:55.374897 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33221aa2-c2cd-4451-9702-6197e8d65e68 service neutron] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:55.433046 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:55.435845 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afce757b-8c25-4b33-b76a-989b6b2eb7f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:55.436650 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afce757b-8c25-4b33-b76a-989b6b2eb7f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:55.437077 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afce757b-8c25-4b33-b76a-989b6b2eb7f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:55.437428 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afce757b-8c25-4b33-b76a-989b6b2eb7f0 admin admin] PATH_INFO: `/v3/users/89884e9165dd49518a110b3c003e85f4` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:55.438775 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afce757b-8c25-4b33-b76a-989b6b2eb7f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89884e9165dd49518a110b3c003e85f4)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:55.440977 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afce757b-8c25-4b33-b76a-989b6b2eb7f0 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:55.568845 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:55.703245 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-afce757b-8c25-4b33-b76a-989b6b2eb7f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89884e9165dd49518a110b3c003e85f4'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:55.716282 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-afce757b-8c25-4b33-b76a-989b6b2eb7f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89884e9165dd49518a110b3c003e85f4'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:55.717045 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-afce757b-8c25-4b33-b76a-989b6b2eb7f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89884e9165dd49518a110b3c003e85f4'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:55.717659 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-afce757b-8c25-4b33-b76a-989b6b2eb7f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89884e9165dd49518a110b3c003e85f4'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:55.809682 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-afce757b-8c25-4b33-b76a-989b6b2eb7f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89884e9165dd49518a110b3c003e85f4'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:55.824132 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1431/2823] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:33:55 2022] DELETE /identity/v3/users/89884e9165dd49518a110b3c003e85f4 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:33:56.318326 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:56.330697 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b51f4d-032b-4598-b192-d5d702e8b677 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:56.332254 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-c1b51f4d-032b-4598-b192-d5d702e8b677 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:56.355500 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b51f4d-032b-4598-b192-d5d702e8b677 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:56.427736 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c2bcd0-f1bd-42b1-8aa1-ffed8201b23a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:56.428743 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c2bcd0-f1bd-42b1-8aa1-ffed8201b23a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:56.429030 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c2bcd0-f1bd-42b1-8aa1-ffed8201b23a admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:56.429529 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c2bcd0-f1bd-42b1-8aa1-ffed8201b23a admin admin] PATH_INFO: `/v3/projects/3428760e9c794d59a9b257b7b8f7aade` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:56.430834 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c2bcd0-f1bd-42b1-8aa1-ffed8201b23a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3428760e9c794d59a9b257b7b8f7aade)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:56.435209 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c2bcd0-f1bd-42b1-8aa1-ffed8201b23a admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:56.437134 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:56.562470 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-70c2bcd0-f1bd-42b1-8aa1-ffed8201b23a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3428760e9c794d59a9b257b7b8f7aade', 'request_id': 'req-70c2bcd0-f1bd-42b1-8aa1-ffed8201b23a'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:56.568687 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-70c2bcd0-f1bd-42b1-8aa1-ffed8201b23a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3428760e9c794d59a9b257b7b8f7aade', 'request_id': 'req-70c2bcd0-f1bd-42b1-8aa1-ffed8201b23a'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:56.653767 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-70c2bcd0-f1bd-42b1-8aa1-ffed8201b23a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3428760e9c794d59a9b257b7b8f7aade', 'request_id': 'req-70c2bcd0-f1bd-42b1-8aa1-ffed8201b23a'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:56.665622 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-70c2bcd0-f1bd-42b1-8aa1-ffed8201b23a admin admin] The token cache is being invalidate because project 3428760e9c794d59a9b257b7b8f7aade was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:33:56.665912 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-70c2bcd0-f1bd-42b1-8aa1-ffed8201b23a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:56.666553 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-70c2bcd0-f1bd-42b1-8aa1-ffed8201b23a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:33:56.668262 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1393/2824] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:33:56 2022] DELETE /identity/v3/projects/3428760e9c794d59a9b257b7b8f7aade => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:33:56.673680 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3533cee5-66a7-4487-bb94-3174b158643e None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:56.674150 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3533cee5-66a7-4487-bb94-3174b158643e None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:56.674472 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3533cee5-66a7-4487-bb94-3174b158643e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:56.699352 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-3533cee5-66a7-4487-bb94-3174b158643e None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:33:56.700701 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-3533cee5-66a7-4487-bb94-3174b158643e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:56.773057 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1432/2825] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:33:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:33:56.780997 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded9c004-c5e7-4819-9bb3-564107fe45df None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:56.781377 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded9c004-c5e7-4819-9bb3-564107fe45df None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:56.784885 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded9c004-c5e7-4819-9bb3-564107fe45df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:56.802493 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-ded9c004-c5e7-4819-9bb3-564107fe45df None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:33:56.803771 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-ded9c004-c5e7-4819-9bb3-564107fe45df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:56.820513 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1394/2826] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:33:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:33:56.820651 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3533cee5-66a7-4487-bb94-3174b158643e None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:56.830248 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3533cee5-66a7-4487-bb94-3174b158643e None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:56.837174 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c2545e-041f-40f2-820b-515188dabbcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:56.837824 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c2545e-041f-40f2-820b-515188dabbcf admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:56.838101 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c2545e-041f-40f2-820b-515188dabbcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:56.838369 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c2545e-041f-40f2-820b-515188dabbcf admin admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:56.839078 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c2545e-041f-40f2-820b-515188dabbcf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:33:56.839406 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c2545e-041f-40f2-820b-515188dabbcf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:56.841178 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c2545e-041f-40f2-820b-515188dabbcf admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:56.854980 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1433/2827] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Aug 4 21:33:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:33:56.862903 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded9c004-c5e7-4819-9bb3-564107fe45df None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:56.875993 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded9c004-c5e7-4819-9bb3-564107fe45df None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:56.881908 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd6cad2-dba4-4b99-8e07-13ac0981ec43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:56.882610 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd6cad2-dba4-4b99-8e07-13ac0981ec43 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:56.882948 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd6cad2-dba4-4b99-8e07-13ac0981ec43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:56.886078 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd6cad2-dba4-4b99-8e07-13ac0981ec43 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:56.886078 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd6cad2-dba4-4b99-8e07-13ac0981ec43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:56.886078 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd6cad2-dba4-4b99-8e07-13ac0981ec43 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:57.031003 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1395/2828] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:33:56 2022] POST /identity/v3/projects => generated 385 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:33:57.033716 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c2545e-041f-40f2-820b-515188dabbcf admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:57.056856 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c2545e-041f-40f2-820b-515188dabbcf admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:57.139057 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8f807b-83f2-409c-a399-7d26de7cdd55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:57.139748 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8f807b-83f2-409c-a399-7d26de7cdd55 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:57.140034 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8f807b-83f2-409c-a399-7d26de7cdd55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:57.140306 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8f807b-83f2-409c-a399-7d26de7cdd55 admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:57.140984 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8f807b-83f2-409c-a399-7d26de7cdd55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:57.142910 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8f807b-83f2-409c-a399-7d26de7cdd55 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:57.249394 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1434/2829] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:33:57 2022] POST /identity/v3/users => generated 293 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:33:57.252063 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd6cad2-dba4-4b99-8e07-13ac0981ec43 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:57.268891 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd6cad2-dba4-4b99-8e07-13ac0981ec43 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:57.274916 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794f4b6d-8c4b-4505-a4cc-19c3b4c1d736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:57.278216 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794f4b6d-8c4b-4505-a4cc-19c3b4c1d736 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:57.278216 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794f4b6d-8c4b-4505-a4cc-19c3b4c1d736 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:57.278216 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794f4b6d-8c4b-4505-a4cc-19c3b4c1d736 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:57.278216 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794f4b6d-8c4b-4505-a4cc-19c3b4c1d736 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:57.278573 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794f4b6d-8c4b-4505-a4cc-19c3b4c1d736 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:57.291361 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1396/2830] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:33:57 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:33:57.295243 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8f807b-83f2-409c-a399-7d26de7cdd55 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:57.306532 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8f807b-83f2-409c-a399-7d26de7cdd55 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:57.311360 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e02a20-ffa2-41ef-859b-b23f7fd5feef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:57.311993 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e02a20-ffa2-41ef-859b-b23f7fd5feef admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:57.312265 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e02a20-ffa2-41ef-859b-b23f7fd5feef admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:57.312545 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e02a20-ffa2-41ef-859b-b23f7fd5feef admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:57.313252 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e02a20-ffa2-41ef-859b-b23f7fd5feef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:57.315348 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e02a20-ffa2-41ef-859b-b23f7fd5feef admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:57.336063 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1435/2831] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:33:57 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:33:57.337500 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794f4b6d-8c4b-4505-a4cc-19c3b4c1d736 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:57.349768 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794f4b6d-8c4b-4505-a4cc-19c3b4c1d736 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:57.362376 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8be39f-744e-4e36-bb05-cd23e454f0ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:57.363880 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8be39f-744e-4e36-bb05-cd23e454f0ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:57.364197 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8be39f-744e-4e36-bb05-cd23e454f0ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:57.364473 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8be39f-744e-4e36-bb05-cd23e454f0ff admin admin] PATH_INFO: `/v3/projects/cc6f69aea13545a7a8756844f35c2961/users/809a4e7b5596438aa85708ecd9f73ead/roles/8feebc60363c4feb870718b1721e5324` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:57.386892 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8be39f-744e-4e36-bb05-cd23e454f0ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc6f69aea13545a7a8756844f35c2961, user_id=809a4e7b5596438aa85708ecd9f73ead, role_id=8feebc60363c4feb870718b1721e5324)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:57.389726 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8be39f-744e-4e36-bb05-cd23e454f0ff admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:57.513382 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:57.519507 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1397/2832] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:33:57 2022] PUT /identity/v3/projects/cc6f69aea13545a7a8756844f35c2961/users/809a4e7b5596438aa85708ecd9f73ead/roles/8feebc60363c4feb870718b1721e5324 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:33:57.532152 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e215de83-19db-4183-9ddd-9b3ed61d3df5 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:57.532725 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e215de83-19db-4183-9ddd-9b3ed61d3df5 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:57.533199 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e215de83-19db-4183-9ddd-9b3ed61d3df5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:57.566659 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-e215de83-19db-4183-9ddd-9b3ed61d3df5 None None] MFA Rules not processed for user `809a4e7b5596438aa85708ecd9f73ead`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:33:57.568502 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-e215de83-19db-4183-9ddd-9b3ed61d3df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:57.613663 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:57.646777 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1436/2833] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:33:57 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:33:57.659218 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8be39f-744e-4e36-bb05-cd23e454f0ff admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:57.661604 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-ca8be39f-744e-4e36-bb05-cd23e454f0ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:57.738355 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8be39f-744e-4e36-bb05-cd23e454f0ff admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:57.742347 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5538ea83-4079-4fec-99d9-ddc714caa4c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e31f2728056b460fa474478eb85ca1d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:57.743287 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5538ea83-4079-4fec-99d9-ddc714caa4c2 service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:57.743642 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5538ea83-4079-4fec-99d9-ddc714caa4c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:57.744422 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5538ea83-4079-4fec-99d9-ddc714caa4c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:57.754924 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5538ea83-4079-4fec-99d9-ddc714caa4c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:57.759687 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5538ea83-4079-4fec-99d9-ddc714caa4c2 service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:57.771133 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1398/2834] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:33:57 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:34:00.921215 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9f3620-c702-4ff6-af27-fd77c2415d15 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:34:00.921215 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9f3620-c702-4ff6-af27-fd77c2415d15 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:34:00.921215 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9f3620-c702-4ff6-af27-fd77c2415d15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:34:00.950964 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-bd9f3620-c702-4ff6-af27-fd77c2415d15 None None] MFA Rules not processed for user `2fc55eab88954503807d531cadd08543`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:34:00.952383 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-bd9f3620-c702-4ff6-af27-fd77c2415d15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:34:01.016251 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5538ea83-4079-4fec-99d9-ddc714caa4c2 service cinder] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:34:01.043381 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1437/2835] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:34:00 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:34:01.048362 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9f3620-c702-4ff6-af27-fd77c2415d15 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:34:01.050015 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-bd9f3620-c702-4ff6-af27-fd77c2415d15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:34:01.099508 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5538ea83-4079-4fec-99d9-ddc714caa4c2 service cinder] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:34:01.108349 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf3c10c-d375-465f-9c1f-56168b4b9092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:34:01.109024 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf3c10c-d375-465f-9c1f-56168b4b9092 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:34:01.109317 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf3c10c-d375-465f-9c1f-56168b4b9092 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:34:01.109595 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf3c10c-d375-465f-9c1f-56168b4b9092 admin admin] PATH_INFO: `/v3/users/809a4e7b5596438aa85708ecd9f73ead` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:34:01.110764 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf3c10c-d375-465f-9c1f-56168b4b9092 admin admin] RBAC: Authorizing `identity:delete_user(user_id=809a4e7b5596438aa85708ecd9f73ead)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:34:01.113468 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf3c10c-d375-465f-9c1f-56168b4b9092 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:34:01.124137 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9f3620-c702-4ff6-af27-fd77c2415d15 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:34:01.130296 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d31ed6-e517-4dc1-b5d8-837ac2ba8868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:34:01.131225 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d31ed6-e517-4dc1-b5d8-837ac2ba8868 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:34:01.132396 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d31ed6-e517-4dc1-b5d8-837ac2ba8868 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:34:01.132981 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d31ed6-e517-4dc1-b5d8-837ac2ba8868 admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:34:01.133958 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d31ed6-e517-4dc1-b5d8-837ac2ba8868 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:34:01.136087 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d31ed6-e517-4dc1-b5d8-837ac2ba8868 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:34:01.216120 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1438/2836] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:34:01 2022] POST /identity/v3/projects => generated 385 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:34:01.220280 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d31ed6-e517-4dc1-b5d8-837ac2ba8868 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:34:01.268138 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-3bf3c10c-d375-465f-9c1f-56168b4b9092 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '809a4e7b5596438aa85708ecd9f73ead'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:34:01.282718 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-3bf3c10c-d375-465f-9c1f-56168b4b9092 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '809a4e7b5596438aa85708ecd9f73ead'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:34:01.283666 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-3bf3c10c-d375-465f-9c1f-56168b4b9092 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '809a4e7b5596438aa85708ecd9f73ead'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:34:01.285791 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=90,keystone:INSERT=1,keystone:DELETE=9 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:34:01.287360 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:34:01.291641 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-3bf3c10c-d375-465f-9c1f-56168b4b9092 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '809a4e7b5596438aa85708ecd9f73ead'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:34:01.292429 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d31ed6-e517-4dc1-b5d8-837ac2ba8868 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:34:01.292586 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-3bf3c10c-d375-465f-9c1f-56168b4b9092 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '809a4e7b5596438aa85708ecd9f73ead'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:34:01.297229 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324e8427-ea4f-4b8d-b4fd-27b3c00cbf80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:34:01.297850 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324e8427-ea4f-4b8d-b4fd-27b3c00cbf80 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:34:01.298182 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324e8427-ea4f-4b8d-b4fd-27b3c00cbf80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:34:01.298411 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324e8427-ea4f-4b8d-b4fd-27b3c00cbf80 admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:34:01.299213 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324e8427-ea4f-4b8d-b4fd-27b3c00cbf80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:34:01.301290 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324e8427-ea4f-4b8d-b4fd-27b3c00cbf80 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:34:01.350176 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1399/2837] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:34:01 2022] DELETE /identity/v3/users/809a4e7b5596438aa85708ecd9f73ead => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:34:01.354547 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e88c26-14e0-44ce-a2c0-dbe03623f25c None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:34:01.354836 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e88c26-14e0-44ce-a2c0-dbe03623f25c None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:34:01.355111 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e88c26-14e0-44ce-a2c0-dbe03623f25c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:34:01.373723 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-d8e88c26-14e0-44ce-a2c0-dbe03623f25c None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:34:01.374900 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-d8e88c26-14e0-44ce-a2c0-dbe03623f25c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:34:01.390435 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1439/2838] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:34:01 2022] POST /identity/v3/users => generated 299 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:34:01.394808 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324e8427-ea4f-4b8d-b4fd-27b3c00cbf80 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:34:01.396802 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-324e8427-ea4f-4b8d-b4fd-27b3c00cbf80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:34:01.418239 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324e8427-ea4f-4b8d-b4fd-27b3c00cbf80 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:34:01.459518 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1400/2839] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:34:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:34:01.469231 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e88c26-14e0-44ce-a2c0-dbe03623f25c None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:34:01.472573 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-d8e88c26-14e0-44ce-a2c0-dbe03623f25c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:34:01.481339 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297b8ff8-f9df-459a-9bdb-18e4f786b051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:34:01.482467 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297b8ff8-f9df-459a-9bdb-18e4f786b051 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:34:01.482830 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297b8ff8-f9df-459a-9bdb-18e4f786b051 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:34:01.483125 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297b8ff8-f9df-459a-9bdb-18e4f786b051 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:34:01.490378 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297b8ff8-f9df-459a-9bdb-18e4f786b051 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:34:01.492103 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297b8ff8-f9df-459a-9bdb-18e4f786b051 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:34:01.507209 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1440/2840] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:34:01 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:34:01.508841 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297b8ff8-f9df-459a-9bdb-18e4f786b051 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:34:01.520585 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297b8ff8-f9df-459a-9bdb-18e4f786b051 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:34:01.525971 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dfd6be-553a-46b0-9c6a-535fcd980228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:34:01.526738 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dfd6be-553a-46b0-9c6a-535fcd980228 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:34:01.527177 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dfd6be-553a-46b0-9c6a-535fcd980228 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:34:01.527557 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dfd6be-553a-46b0-9c6a-535fcd980228 admin admin] PATH_INFO: `/v3/projects/05f83096d12840719d38b4bc6328dfc1/users/d15ea8aac7b747a788d153bfcc629ea6/roles/a432dd4735944ecfab2fa5ce1705d356` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:34:01.543347 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e88c26-14e0-44ce-a2c0-dbe03623f25c None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:34:01.543593 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dfd6be-553a-46b0-9c6a-535fcd980228 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05f83096d12840719d38b4bc6328dfc1, user_id=d15ea8aac7b747a788d153bfcc629ea6, role_id=a432dd4735944ecfab2fa5ce1705d356)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:34:01.546270 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dfd6be-553a-46b0-9c6a-535fcd980228 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:34:01.548208 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be74f4d5-cdc9-4cbe-9b86-e229b1711d86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:34:01.549467 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be74f4d5-cdc9-4cbe-9b86-e229b1711d86 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:34:01.550693 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be74f4d5-cdc9-4cbe-9b86-e229b1711d86 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:34:01.551515 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be74f4d5-cdc9-4cbe-9b86-e229b1711d86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:34:01.564857 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be74f4d5-cdc9-4cbe-9b86-e229b1711d86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:34:01.566953 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be74f4d5-cdc9-4cbe-9b86-e229b1711d86 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:34:01.576732 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1401/2841] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:34:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:34:01.614847 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1441/2842] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:34:01 2022] PUT /identity/v3/projects/05f83096d12840719d38b4bc6328dfc1/users/d15ea8aac7b747a788d153bfcc629ea6/roles/a432dd4735944ecfab2fa5ce1705d356 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:34:01.618905 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be74f4d5-cdc9-4cbe-9b86-e229b1711d86 service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:34:01.629545 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be74f4d5-cdc9-4cbe-9b86-e229b1711d86 service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:34:01.698876 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932e3e09-2cea-4663-a6c3-f214fb786da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:34:01.699560 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932e3e09-2cea-4663-a6c3-f214fb786da1 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:34:01.699848 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932e3e09-2cea-4663-a6c3-f214fb786da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:34:01.700123 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932e3e09-2cea-4663-a6c3-f214fb786da1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:34:01.700818 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932e3e09-2cea-4663-a6c3-f214fb786da1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:34:01.702444 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932e3e09-2cea-4663-a6c3-f214fb786da1 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:34:01.712652 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1402/2843] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:34:01 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:34:01.717480 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dfd6be-553a-46b0-9c6a-535fcd980228 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:34:01.729628 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dfd6be-553a-46b0-9c6a-535fcd980228 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:34:01.734589 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f89cf1-60e3-4a31-b520-8ebff7271024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:34:01.735269 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f89cf1-60e3-4a31-b520-8ebff7271024 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:34:01.735585 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f89cf1-60e3-4a31-b520-8ebff7271024 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:34:01.735865 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f89cf1-60e3-4a31-b520-8ebff7271024 admin admin] PATH_INFO: `/v3/projects/05f83096d12840719d38b4bc6328dfc1/users/d15ea8aac7b747a788d153bfcc629ea6/roles/a432dd4735944ecfab2fa5ce1705d356` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:34:01.738379 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f89cf1-60e3-4a31-b520-8ebff7271024 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05f83096d12840719d38b4bc6328dfc1, user_id=d15ea8aac7b747a788d153bfcc629ea6, role_id=a432dd4735944ecfab2fa5ce1705d356)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:34:01.741115 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f89cf1-60e3-4a31-b520-8ebff7271024 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:34:01.752621 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1442/2844] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:34:01 2022] PUT /identity/v3/projects/05f83096d12840719d38b4bc6328dfc1/users/d15ea8aac7b747a788d153bfcc629ea6/roles/a432dd4735944ecfab2fa5ce1705d356 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:34:02.061411 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932e3e09-2cea-4663-a6c3-f214fb786da1 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:34:02.063296 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-932e3e09-2cea-4663-a6c3-f214fb786da1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:34:02.092725 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932e3e09-2cea-4663-a6c3-f214fb786da1 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:34:02.135531 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:34:02.209859 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd921feb-4a76-4343-a665-8f26a21df0b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:34:02.209859 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd921feb-4a76-4343-a665-8f26a21df0b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:34:02.209859 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd921feb-4a76-4343-a665-8f26a21df0b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:34:02.209859 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd921feb-4a76-4343-a665-8f26a21df0b7 admin admin] PATH_INFO: `/v3/projects/cc6f69aea13545a7a8756844f35c2961` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:34:02.211020 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd921feb-4a76-4343-a665-8f26a21df0b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc6f69aea13545a7a8756844f35c2961)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:34:02.213356 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd921feb-4a76-4343-a665-8f26a21df0b7 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:34:02.458999 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-bd921feb-4a76-4343-a665-8f26a21df0b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc6f69aea13545a7a8756844f35c2961', 'request_id': 'req-bd921feb-4a76-4343-a665-8f26a21df0b7'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:34:02.469913 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-bd921feb-4a76-4343-a665-8f26a21df0b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc6f69aea13545a7a8756844f35c2961', 'request_id': 'req-bd921feb-4a76-4343-a665-8f26a21df0b7'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:34:02.543719 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-bd921feb-4a76-4343-a665-8f26a21df0b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc6f69aea13545a7a8756844f35c2961', 'request_id': 'req-bd921feb-4a76-4343-a665-8f26a21df0b7'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:34:02.559516 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-bd921feb-4a76-4343-a665-8f26a21df0b7 admin admin] The token cache is being invalidate because project cc6f69aea13545a7a8756844f35c2961 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:34:02.559764 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-bd921feb-4a76-4343-a665-8f26a21df0b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:34:02.560417 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-bd921feb-4a76-4343-a665-8f26a21df0b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:34:02.561955 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1403/2845] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:34:02 2022] DELETE /identity/v3/projects/cc6f69aea13545a7a8756844f35c2961 => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:34:07.638374 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dd2c9f-f1bc-4b42-9d0a-0a62cde677b0 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:34:07.638928 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dd2c9f-f1bc-4b42-9d0a-0a62cde677b0 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:34:07.638928 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dd2c9f-f1bc-4b42-9d0a-0a62cde677b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:34:07.680790 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-82dd2c9f-f1bc-4b42-9d0a-0a62cde677b0 None None] MFA Rules not processed for user `d15ea8aac7b747a788d153bfcc629ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:34:07.682020 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-82dd2c9f-f1bc-4b42-9d0a-0a62cde677b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:34:07.754701 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1443/2846] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:34:07 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:34:07.766590 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd921feb-4a76-4343-a665-8f26a21df0b7 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:34:07.768261 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-bd921feb-4a76-4343-a665-8f26a21df0b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:34:07.830822 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd921feb-4a76-4343-a665-8f26a21df0b7 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:34:07.834573 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2851db3b-8664-4f26-b58d-0939bfc3bcc8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6885c7d929f4f8e96d3890724c3b53f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:34:07.835193 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2851db3b-8664-4f26-b58d-0939bfc3bcc8 service glance] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:34:07.835506 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2851db3b-8664-4f26-b58d-0939bfc3bcc8 service glance] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:34:07.835781 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2851db3b-8664-4f26-b58d-0939bfc3bcc8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:34:07.837511 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-2851db3b-8664-4f26-b58d-0939bfc3bcc8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:34:07.897411 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2851db3b-8664-4f26-b58d-0939bfc3bcc8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:34:07.898849 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2851db3b-8664-4f26-b58d-0939bfc3bcc8 service glance] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:34:07.907773 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1404/2847] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:34:07 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:34:08.189764 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4601b79f-17ee-4368-8dbf-96bd82ac1e78 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:34:08.189764 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4601b79f-17ee-4368-8dbf-96bd82ac1e78 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:34:08.189764 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4601b79f-17ee-4368-8dbf-96bd82ac1e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:34:08.209605 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-4601b79f-17ee-4368-8dbf-96bd82ac1e78 None None] MFA Rules not processed for user `23ea380385f649628cc27e3603bc700c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:34:08.210829 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-4601b79f-17ee-4368-8dbf-96bd82ac1e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:34:08.254268 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:34:08.292285 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1444/2848] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Aug 4 21:34:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:34:08.440120 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2851db3b-8664-4f26-b58d-0939bfc3bcc8 service glance] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:34:08.442192 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-2851db3b-8664-4f26-b58d-0939bfc3bcc8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:34:08.453339 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:34:08.536457 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2851db3b-8664-4f26-b58d-0939bfc3bcc8 service glance] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:34:08.541838 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c325f1d-1415-4235-99c9-b662edead2a1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58935367c0ad445f8477cdac0df04d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:34:08.542598 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c325f1d-1415-4235-99c9-b662edead2a1 service swift] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:34:08.542956 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c325f1d-1415-4235-99c9-b662edead2a1 service swift] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:34:08.548178 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c325f1d-1415-4235-99c9-b662edead2a1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:34:08.553853 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c325f1d-1415-4235-99c9-b662edead2a1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:34:08.555727 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c325f1d-1415-4235-99c9-b662edead2a1 service swift] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:34:08.565255 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1405/2849] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Aug 4 21:34:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 21:34:18.430287 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=12 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:34:18.758222 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c463ae-7687-4910-9a12-bd5c8f158b2e None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:34:18.758520 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c463ae-7687-4910-9a12-bd5c8f158b2e None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:34:18.758800 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c463ae-7687-4910-9a12-bd5c8f158b2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:34:18.778468 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-61c463ae-7687-4910-9a12-bd5c8f158b2e None None] MFA Rules not processed for user `2fc55eab88954503807d531cadd08543`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:34:18.779744 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-61c463ae-7687-4910-9a12-bd5c8f158b2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:34:18.787333 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1445/2850] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:34:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:34:18.796182 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=28 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:34:25.040534 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a188e0a-841e-49e5-9186-73414430caf4 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:34:25.041238 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a188e0a-841e-49e5-9186-73414430caf4 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:34:25.041238 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a188e0a-841e-49e5-9186-73414430caf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:34:25.061196 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-3a188e0a-841e-49e5-9186-73414430caf4 None None] MFA Rules not processed for user `23ea380385f649628cc27e3603bc700c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:34:25.062590 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-3a188e0a-841e-49e5-9186-73414430caf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:34:25.071069 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1406/2851] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Aug 4 21:34:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:34:25.086051 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c463ae-7687-4910-9a12-bd5c8f158b2e None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:34:25.099209 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c463ae-7687-4910-9a12-bd5c8f158b2e None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:34:25.105873 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722c5af0-a261-45bc-8115-e7bc8b2d6721 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58935367c0ad445f8477cdac0df04d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:34:25.106498 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722c5af0-a261-45bc-8115-e7bc8b2d6721 service swift] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:34:25.106781 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722c5af0-a261-45bc-8115-e7bc8b2d6721 service swift] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:34:25.107053 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722c5af0-a261-45bc-8115-e7bc8b2d6721 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:34:25.115505 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722c5af0-a261-45bc-8115-e7bc8b2d6721 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:34:25.117100 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722c5af0-a261-45bc-8115-e7bc8b2d6721 service swift] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:34:25.125367 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1446/2852] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Aug 4 21:34:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 21:34:35.056189 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:34:35.121975 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=9 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:34:53.355240 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269c0e8a-73d2-4384-b592-67f8945b4377 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:34:53.355808 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269c0e8a-73d2-4384-b592-67f8945b4377 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:34:53.355902 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269c0e8a-73d2-4384-b592-67f8945b4377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:34:53.375882 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-269c0e8a-73d2-4384-b592-67f8945b4377 None None] MFA Rules not processed for user `d1946505fcad422ba5416834ad66050d`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:34:53.377098 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-269c0e8a-73d2-4384-b592-67f8945b4377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:34:53.433543 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1407/2853] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:34:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:34:57.254700 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722c5af0-a261-45bc-8115-e7bc8b2d6721 service swift] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:34:57.256352 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-722c5af0-a261-45bc-8115-e7bc8b2d6721 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:34:57.314585 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722c5af0-a261-45bc-8115-e7bc8b2d6721 service swift] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:34:57.318286 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca66c96-9a0b-46da-852a-61d048281a17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:34:57.318906 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca66c96-9a0b-46da-852a-61d048281a17 service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:34:57.319190 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca66c96-9a0b-46da-852a-61d048281a17 service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:34:57.319507 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca66c96-9a0b-46da-852a-61d048281a17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:34:57.329991 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca66c96-9a0b-46da-852a-61d048281a17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:34:57.331607 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca66c96-9a0b-46da-852a-61d048281a17 service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:34:57.343306 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1447/2854] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:34:57 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:35:03.426180 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=25 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:35:06.220604 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269c0e8a-73d2-4384-b592-67f8945b4377 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:35:06.222206 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-269c0e8a-73d2-4384-b592-67f8945b4377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:35:06.235858 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269c0e8a-73d2-4384-b592-67f8945b4377 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:35:06.240747 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d6b808-391e-465c-929c-b87ee3c91133 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:35:06.241371 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d6b808-391e-465c-929c-b87ee3c91133 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:35:06.241652 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d6b808-391e-465c-929c-b87ee3c91133 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:35:06.241922 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d6b808-391e-465c-929c-b87ee3c91133 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:35:06.243733 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-19d6b808-391e-465c-929c-b87ee3c91133 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:35:06.315727 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d6b808-391e-465c-929c-b87ee3c91133 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:35:06.317181 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d6b808-391e-465c-929c-b87ee3c91133 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:35:06.329087 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1408/2855] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:35:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:35:07.337799 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=26 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:35:16.324010 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=34 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:35:18.311049 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca66c96-9a0b-46da-852a-61d048281a17 service neutron] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:35:18.319602 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca66c96-9a0b-46da-852a-61d048281a17 service neutron] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:35:18.323149 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d3ee03-066c-4e3c-9cd2-23592451c71c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:35:18.324050 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d3ee03-066c-4e3c-9cd2-23592451c71c service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:35:18.324327 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d3ee03-066c-4e3c-9cd2-23592451c71c service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:35:18.324604 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d3ee03-066c-4e3c-9cd2-23592451c71c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:35:18.326352 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-d8d3ee03-066c-4e3c-9cd2-23592451c71c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:35:18.341882 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d3ee03-066c-4e3c-9cd2-23592451c71c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:35:18.344771 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d3ee03-066c-4e3c-9cd2-23592451c71c service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:35:18.357893 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1448/2856] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:35:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:35:28.352088 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=6 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:35:34.169233 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488999b4-e2c8-4438-8ad7-8ca2253a31c9 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:35:34.169709 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488999b4-e2c8-4438-8ad7-8ca2253a31c9 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:35:34.169813 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488999b4-e2c8-4438-8ad7-8ca2253a31c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:35:34.191560 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-488999b4-e2c8-4438-8ad7-8ca2253a31c9 None None] MFA Rules not processed for user `b665d491470f4a0b95c13f51e19d2046`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:35:34.192765 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-488999b4-e2c8-4438-8ad7-8ca2253a31c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:35:34.255250 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1409/2857] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:35:34 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:35:35.519673 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d3ee03-066c-4e3c-9cd2-23592451c71c service neutron] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:35:35.521979 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-d8d3ee03-066c-4e3c-9cd2-23592451c71c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:35:35.617891 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d3ee03-066c-4e3c-9cd2-23592451c71c service neutron] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:35:35.623112 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d5c76b-9da5-4986-a738-39b32850e9b9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67d44a0098ab4c36b14b36e9e59ae75c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:35:35.623783 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d5c76b-9da5-4986-a738-39b32850e9b9 service placement] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:35:35.624067 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d5c76b-9da5-4986-a738-39b32850e9b9 service placement] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:35:35.624344 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d5c76b-9da5-4986-a738-39b32850e9b9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:35:35.628034 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-22d5c76b-9da5-4986-a738-39b32850e9b9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:35:35.644124 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d5c76b-9da5-4986-a738-39b32850e9b9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:35:35.645838 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d5c76b-9da5-4986-a738-39b32850e9b9 service placement] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:35:35.659259 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1449/2858] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:35:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:35:41.092411 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488999b4-e2c8-4438-8ad7-8ca2253a31c9 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:35:41.102847 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488999b4-e2c8-4438-8ad7-8ca2253a31c9 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:35:41.106630 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33cd95f-d889-43a4-841f-a341c41d7ac1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:35:41.107270 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33cd95f-d889-43a4-841f-a341c41d7ac1 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:35:41.107588 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33cd95f-d889-43a4-841f-a341c41d7ac1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:35:41.107860 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33cd95f-d889-43a4-841f-a341c41d7ac1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:35:41.109767 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-b33cd95f-d889-43a4-841f-a341c41d7ac1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:35:41.126173 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33cd95f-d889-43a4-841f-a341c41d7ac1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:35:41.126173 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33cd95f-d889-43a4-841f-a341c41d7ac1 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:35:41.132573 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1410/2859] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:35:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:35:41.943768 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d5c76b-9da5-4986-a738-39b32850e9b9 service placement] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:35:41.943768 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-22d5c76b-9da5-4986-a738-39b32850e9b9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:35:41.963417 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d5c76b-9da5-4986-a738-39b32850e9b9 service placement] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:35:41.972544 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fa3825-f5b4-425d-950a-a78a75b17579 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a2ce1df87fa47e5be301916bf3245fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:35:41.973576 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fa3825-f5b4-425d-950a-a78a75b17579 service nova] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:35:41.974073 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fa3825-f5b4-425d-950a-a78a75b17579 service nova] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:35:41.975194 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fa3825-f5b4-425d-950a-a78a75b17579 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:35:41.978293 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-15fa3825-f5b4-425d-950a-a78a75b17579 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:35:42.000551 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fa3825-f5b4-425d-950a-a78a75b17579 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:35:42.002355 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fa3825-f5b4-425d-950a-a78a75b17579 service nova] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:35:42.017835 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1450/2860] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:35:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:35:51.126867 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=31 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:35:52.010808 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=38 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:35:58.774164 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33cd95f-d889-43a4-841f-a341c41d7ac1 service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:35:58.788171 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33cd95f-d889-43a4-841f-a341c41d7ac1 service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:35:58.794449 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bfafe4-3737-473e-b7d4-7c0fa3009e28 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a2ce1df87fa47e5be301916bf3245fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:35:58.795100 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bfafe4-3737-473e-b7d4-7c0fa3009e28 service nova] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:35:58.795409 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bfafe4-3737-473e-b7d4-7c0fa3009e28 service nova] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:35:58.795686 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bfafe4-3737-473e-b7d4-7c0fa3009e28 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:35:58.797545 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-e6bfafe4-3737-473e-b7d4-7c0fa3009e28 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:35:58.812716 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bfafe4-3737-473e-b7d4-7c0fa3009e28 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:35:58.814311 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bfafe4-3737-473e-b7d4-7c0fa3009e28 service nova] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:35:58.832620 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1411/2861] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:35:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:36:01.641737 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fa3825-f5b4-425d-950a-a78a75b17579 service nova] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:36:01.653670 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fa3825-f5b4-425d-950a-a78a75b17579 service nova] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:36:01.658568 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d4cffa-8b1d-4207-9e92-64e663cfd28f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67d44a0098ab4c36b14b36e9e59ae75c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:36:01.662634 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d4cffa-8b1d-4207-9e92-64e663cfd28f service placement] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:36:01.662634 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d4cffa-8b1d-4207-9e92-64e663cfd28f service placement] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:36:01.662634 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d4cffa-8b1d-4207-9e92-64e663cfd28f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:36:01.662634 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-44d4cffa-8b1d-4207-9e92-64e663cfd28f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:36:01.674491 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d4cffa-8b1d-4207-9e92-64e663cfd28f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:36:01.676116 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d4cffa-8b1d-4207-9e92-64e663cfd28f service placement] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:36:01.687755 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1451/2862] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:36:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:36:01.766497 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bfafe4-3737-473e-b7d4-7c0fa3009e28 service nova] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:36:01.768083 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-e6bfafe4-3737-473e-b7d4-7c0fa3009e28 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:36:01.833962 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bfafe4-3737-473e-b7d4-7c0fa3009e28 service nova] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:36:01.838528 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49ec122-27dc-4c78-99fc-c3fc93960bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:36:01.839186 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49ec122-27dc-4c78-99fc-c3fc93960bc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:36:01.839510 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49ec122-27dc-4c78-99fc-c3fc93960bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:36:01.839787 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49ec122-27dc-4c78-99fc-c3fc93960bc5 admin admin] PATH_INFO: `/v3/users/d1946505fcad422ba5416834ad66050d` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:36:01.840926 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49ec122-27dc-4c78-99fc-c3fc93960bc5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1946505fcad422ba5416834ad66050d)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:36:01.842912 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49ec122-27dc-4c78-99fc-c3fc93960bc5 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:36:02.054170 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-d49ec122-27dc-4c78-99fc-c3fc93960bc5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1946505fcad422ba5416834ad66050d'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:36:02.065861 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-d49ec122-27dc-4c78-99fc-c3fc93960bc5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1946505fcad422ba5416834ad66050d'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:36:02.066518 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-d49ec122-27dc-4c78-99fc-c3fc93960bc5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1946505fcad422ba5416834ad66050d'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:36:02.073701 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-d49ec122-27dc-4c78-99fc-c3fc93960bc5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1946505fcad422ba5416834ad66050d'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:36:02.074608 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-d49ec122-27dc-4c78-99fc-c3fc93960bc5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1946505fcad422ba5416834ad66050d'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:36:02.159267 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1412/2863] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:36:01 2022] DELETE /identity/v3/users/d1946505fcad422ba5416834ad66050d => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:36:02.430528 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d4cffa-8b1d-4207-9e92-64e663cfd28f service placement] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:36:02.443782 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d4cffa-8b1d-4207-9e92-64e663cfd28f service placement] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:36:02.510200 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9262f40a-e751-49e8-a195-d39370532c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:36:02.510840 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9262f40a-e751-49e8-a195-d39370532c3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:36:02.511123 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9262f40a-e751-49e8-a195-d39370532c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:36:02.511436 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9262f40a-e751-49e8-a195-d39370532c3c admin admin] PATH_INFO: `/v3/projects/76ca8c859a9c4762afbcf40fb071c188` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:36:02.512589 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9262f40a-e751-49e8-a195-d39370532c3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=76ca8c859a9c4762afbcf40fb071c188)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:36:02.514576 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9262f40a-e751-49e8-a195-d39370532c3c admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:36:02.606215 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-9262f40a-e751-49e8-a195-d39370532c3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76ca8c859a9c4762afbcf40fb071c188', 'request_id': 'req-9262f40a-e751-49e8-a195-d39370532c3c'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:36:02.611674 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-9262f40a-e751-49e8-a195-d39370532c3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76ca8c859a9c4762afbcf40fb071c188', 'request_id': 'req-9262f40a-e751-49e8-a195-d39370532c3c'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:36:02.674238 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-9262f40a-e751-49e8-a195-d39370532c3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76ca8c859a9c4762afbcf40fb071c188', 'request_id': 'req-9262f40a-e751-49e8-a195-d39370532c3c'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:36:02.687002 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-9262f40a-e751-49e8-a195-d39370532c3c admin admin] The token cache is being invalidate because project 76ca8c859a9c4762afbcf40fb071c188 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:36:02.687291 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-9262f40a-e751-49e8-a195-d39370532c3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:36:02.687986 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-9262f40a-e751-49e8-a195-d39370532c3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:36:02.689583 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1452/2864] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:36:02 2022] DELETE /identity/v3/projects/76ca8c859a9c4762afbcf40fb071c188 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:36:02.694416 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26834838-82ff-4b65-8702-328a244bd8eb None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:36:02.694711 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26834838-82ff-4b65-8702-328a244bd8eb None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:36:02.694984 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26834838-82ff-4b65-8702-328a244bd8eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:36:02.715139 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-26834838-82ff-4b65-8702-328a244bd8eb None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:36:02.716380 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-26834838-82ff-4b65-8702-328a244bd8eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:36:02.774531 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1413/2865] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:36:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:36:02.783007 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e17fd0b-54a7-4afa-a434-b1fe5a3cd821 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:36:02.783335 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e17fd0b-54a7-4afa-a434-b1fe5a3cd821 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:36:02.783631 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e17fd0b-54a7-4afa-a434-b1fe5a3cd821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:36:02.803275 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-1e17fd0b-54a7-4afa-a434-b1fe5a3cd821 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:36:02.804509 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-1e17fd0b-54a7-4afa-a434-b1fe5a3cd821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:36:02.812478 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1453/2866] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:36:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:36:02.816761 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26834838-82ff-4b65-8702-328a244bd8eb None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:36:02.826528 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26834838-82ff-4b65-8702-328a244bd8eb None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:36:02.831071 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4bed85-543a-4bef-b87e-950669c6305e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:36:02.831715 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4bed85-543a-4bef-b87e-950669c6305e admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:36:02.831993 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4bed85-543a-4bef-b87e-950669c6305e admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:36:02.832263 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4bed85-543a-4bef-b87e-950669c6305e admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:36:02.832953 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4bed85-543a-4bef-b87e-950669c6305e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:36:02.833300 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4bed85-543a-4bef-b87e-950669c6305e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:36:02.834879 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4bed85-543a-4bef-b87e-950669c6305e admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:36:02.836324 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:36:02.847115 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1414/2867] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Aug 4 21:36:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:36:02.854479 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e17fd0b-54a7-4afa-a434-b1fe5a3cd821 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:36:02.863976 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e17fd0b-54a7-4afa-a434-b1fe5a3cd821 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:36:02.868729 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71a1f1c-10ca-4edc-8d8f-091a22668f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:36:02.869335 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71a1f1c-10ca-4edc-8d8f-091a22668f18 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:36:02.869639 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71a1f1c-10ca-4edc-8d8f-091a22668f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:36:02.869918 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71a1f1c-10ca-4edc-8d8f-091a22668f18 admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:36:02.870600 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71a1f1c-10ca-4edc-8d8f-091a22668f18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:36:02.872574 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71a1f1c-10ca-4edc-8d8f-091a22668f18 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:36:02.938715 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1454/2868] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Aug 4 21:36:02 2022] POST /identity/v3/projects => generated 387 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:36:02.942831 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4bed85-543a-4bef-b87e-950669c6305e admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:36:02.952967 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4bed85-543a-4bef-b87e-950669c6305e admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:36:03.008925 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d73be4-227f-4b3c-93af-e081b7727a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:36:03.009585 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d73be4-227f-4b3c-93af-e081b7727a8c admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:36:03.009870 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d73be4-227f-4b3c-93af-e081b7727a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:36:03.010148 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d73be4-227f-4b3c-93af-e081b7727a8c admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:36:03.010805 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d73be4-227f-4b3c-93af-e081b7727a8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:36:03.012767 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d73be4-227f-4b3c-93af-e081b7727a8c admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:36:03.120365 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1415/2869] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Aug 4 21:36:02 2022] POST /identity/v3/users => generated 294 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:36:03.124629 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71a1f1c-10ca-4edc-8d8f-091a22668f18 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:36:03.135514 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71a1f1c-10ca-4edc-8d8f-091a22668f18 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:36:03.140108 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b98d4c-c86b-4567-afa2-4b211d8c81ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:36:03.140712 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b98d4c-c86b-4567-afa2-4b211d8c81ae admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:36:03.140990 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b98d4c-c86b-4567-afa2-4b211d8c81ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:36:03.141390 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b98d4c-c86b-4567-afa2-4b211d8c81ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:36:03.142084 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b98d4c-c86b-4567-afa2-4b211d8c81ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:36:03.143705 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b98d4c-c86b-4567-afa2-4b211d8c81ae admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:36:03.154260 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1455/2870] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:36:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:36:03.158497 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d73be4-227f-4b3c-93af-e081b7727a8c admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:36:03.168379 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d73be4-227f-4b3c-93af-e081b7727a8c admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:36:03.173192 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8d83b9-cd67-4986-ada8-bd1b2560cf6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:36:03.173895 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8d83b9-cd67-4986-ada8-bd1b2560cf6f admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:36:03.174258 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8d83b9-cd67-4986-ada8-bd1b2560cf6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:36:03.174614 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8d83b9-cd67-4986-ada8-bd1b2560cf6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:36:03.175309 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8d83b9-cd67-4986-ada8-bd1b2560cf6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:36:03.176939 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8d83b9-cd67-4986-ada8-bd1b2560cf6f admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:36:03.188301 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1416/2871] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Aug 4 21:36:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:36:03.192434 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b98d4c-c86b-4567-afa2-4b211d8c81ae admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:36:03.203560 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b98d4c-c86b-4567-afa2-4b211d8c81ae admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:36:03.208322 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c3dc57-09d0-48f8-bfcf-ccc288fbd54f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:36:03.208988 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c3dc57-09d0-48f8-bfcf-ccc288fbd54f admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:36:03.209277 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c3dc57-09d0-48f8-bfcf-ccc288fbd54f admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:36:03.209558 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c3dc57-09d0-48f8-bfcf-ccc288fbd54f admin admin] PATH_INFO: `/v3/projects/ee992b33bf394700b927a306907bdced/users/1873cf76ba1244368af7dd8c11fcfec8/roles/8feebc60363c4feb870718b1721e5324` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:36:03.227518 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c3dc57-09d0-48f8-bfcf-ccc288fbd54f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee992b33bf394700b927a306907bdced, user_id=1873cf76ba1244368af7dd8c11fcfec8, role_id=8feebc60363c4feb870718b1721e5324)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:36:03.230413 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c3dc57-09d0-48f8-bfcf-ccc288fbd54f admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:36:03.285868 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1456/2872] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Aug 4 21:36:03 2022] PUT /identity/v3/projects/ee992b33bf394700b927a306907bdced/users/1873cf76ba1244368af7dd8c11fcfec8/roles/8feebc60363c4feb870718b1721e5324 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:36:03.297093 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b902e653-ecef-47bc-b4b2-453525aebef2 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:36:03.297381 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b902e653-ecef-47bc-b4b2-453525aebef2 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:36:03.297679 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b902e653-ecef-47bc-b4b2-453525aebef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:36:03.325585 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-b902e653-ecef-47bc-b4b2-453525aebef2 None None] MFA Rules not processed for user `1873cf76ba1244368af7dd8c11fcfec8`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:36:03.326796 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-b902e653-ecef-47bc-b4b2-453525aebef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:36:03.390489 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1417/2873] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:36:03 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:36:03.401842 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c3dc57-09d0-48f8-bfcf-ccc288fbd54f admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:36:03.403588 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-74c3dc57-09d0-48f8-bfcf-ccc288fbd54f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:36:03.417898 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:36:03.461239 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c3dc57-09d0-48f8-bfcf-ccc288fbd54f admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:36:03.465389 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4570b07f-8a31-4db3-bfa3-7fabbc75eb1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e31f2728056b460fa474478eb85ca1d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:36:03.466000 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4570b07f-8a31-4db3-bfa3-7fabbc75eb1b service cinder] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:36:03.466282 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4570b07f-8a31-4db3-bfa3-7fabbc75eb1b service cinder] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:36:03.466555 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4570b07f-8a31-4db3-bfa3-7fabbc75eb1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:36:03.475919 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4570b07f-8a31-4db3-bfa3-7fabbc75eb1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:36:03.477357 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4570b07f-8a31-4db3-bfa3-7fabbc75eb1b service cinder] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:36:03.486305 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1457/2874] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:36:03 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:36:13.387496 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=69,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:36:13.497260 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=20 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:36:21.725441 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89f0f71-e2ee-43e8-926f-844e22cf8a91 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:36:21.725935 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89f0f71-e2ee-43e8-926f-844e22cf8a91 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:36:21.726035 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89f0f71-e2ee-43e8-926f-844e22cf8a91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:36:21.775286 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-c89f0f71-e2ee-43e8-926f-844e22cf8a91 None None] MFA Rules not processed for user `23ea380385f649628cc27e3603bc700c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:36:21.776579 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-c89f0f71-e2ee-43e8-926f-844e22cf8a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:36:21.840697 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1418/2875] 10.222.0.25 () {58 vars in 967 bytes} [Thu Aug 4 21:36:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:36:21.880498 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4570b07f-8a31-4db3-bfa3-7fabbc75eb1b service cinder] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:36:21.881951 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-4570b07f-8a31-4db3-bfa3-7fabbc75eb1b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:36:21.974429 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4570b07f-8a31-4db3-bfa3-7fabbc75eb1b service cinder] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:36:21.979527 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099d565e-284b-4eee-95e4-e6f28cba4d12 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58935367c0ad445f8477cdac0df04d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:36:21.980143 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099d565e-284b-4eee-95e4-e6f28cba4d12 service swift] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:36:21.980427 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099d565e-284b-4eee-95e4-e6f28cba4d12 service swift] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:36:21.980702 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099d565e-284b-4eee-95e4-e6f28cba4d12 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:36:21.988804 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099d565e-284b-4eee-95e4-e6f28cba4d12 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:36:21.990389 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099d565e-284b-4eee-95e4-e6f28cba4d12 service swift] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:36:21.998399 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1458/2876] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Aug 4 21:36:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 21:36:25.074092 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89f0f71-e2ee-43e8-926f-844e22cf8a91 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:36:25.149534 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89f0f71-e2ee-43e8-926f-844e22cf8a91 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:36:25.154387 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775dc682-e6e0-4f36-bf24-c2cddf2c68b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:36:25.156226 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775dc682-e6e0-4f36-bf24-c2cddf2c68b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:36:25.156521 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775dc682-e6e0-4f36-bf24-c2cddf2c68b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:36:25.156798 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775dc682-e6e0-4f36-bf24-c2cddf2c68b3 admin admin] PATH_INFO: `/v3/users/1873cf76ba1244368af7dd8c11fcfec8` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:36:25.158040 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775dc682-e6e0-4f36-bf24-c2cddf2c68b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1873cf76ba1244368af7dd8c11fcfec8)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:36:25.160088 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775dc682-e6e0-4f36-bf24-c2cddf2c68b3 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:36:25.331137 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-775dc682-e6e0-4f36-bf24-c2cddf2c68b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1873cf76ba1244368af7dd8c11fcfec8'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:36:25.352347 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-775dc682-e6e0-4f36-bf24-c2cddf2c68b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1873cf76ba1244368af7dd8c11fcfec8'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:36:25.353091 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-775dc682-e6e0-4f36-bf24-c2cddf2c68b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1873cf76ba1244368af7dd8c11fcfec8'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:36:25.363269 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-775dc682-e6e0-4f36-bf24-c2cddf2c68b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1873cf76ba1244368af7dd8c11fcfec8'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:36:25.364779 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-775dc682-e6e0-4f36-bf24-c2cddf2c68b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1873cf76ba1244368af7dd8c11fcfec8'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:36:25.367921 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:36:25.474548 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1419/2877] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:36:25 2022] DELETE /identity/v3/users/1873cf76ba1244368af7dd8c11fcfec8 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:36:25.479036 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578b36b7-112e-4295-a0eb-233da7772a51 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:36:25.479570 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578b36b7-112e-4295-a0eb-233da7772a51 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:36:25.479570 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578b36b7-112e-4295-a0eb-233da7772a51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:36:25.499091 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-578b36b7-112e-4295-a0eb-233da7772a51 None None] MFA Rules not processed for user `bc6bad33dfb24427b1ccf30de722b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:36:25.500470 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-578b36b7-112e-4295-a0eb-233da7772a51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:36:25.560998 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1459/2878] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:36:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:36:25.570686 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775dc682-e6e0-4f36-bf24-c2cddf2c68b3 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:36:25.572165 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-775dc682-e6e0-4f36-bf24-c2cddf2c68b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:36:25.630197 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775dc682-e6e0-4f36-bf24-c2cddf2c68b3 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:36:25.633776 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4e7149-b9e3-4137-9ada-2943ae743198 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:36:25.634409 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e7149-b9e3-4137-9ada-2943ae743198 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:36:25.634692 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e7149-b9e3-4137-9ada-2943ae743198 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:36:25.634970 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e7149-b9e3-4137-9ada-2943ae743198 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:36:25.645180 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4e7149-b9e3-4137-9ada-2943ae743198 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:36:25.646713 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4e7149-b9e3-4137-9ada-2943ae743198 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:36:25.657523 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1420/2879] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:36:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:36:26.202213 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578b36b7-112e-4295-a0eb-233da7772a51 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:36:26.206827 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-578b36b7-112e-4295-a0eb-233da7772a51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:36:26.218534 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578b36b7-112e-4295-a0eb-233da7772a51 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:36:26.225225 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82915f9e-582e-4684-8804-e6228f4680b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:36:26.229389 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82915f9e-582e-4684-8804-e6228f4680b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:36:26.229389 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82915f9e-582e-4684-8804-e6228f4680b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:36:26.229389 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82915f9e-582e-4684-8804-e6228f4680b7 admin admin] PATH_INFO: `/v3/projects/ee992b33bf394700b927a306907bdced` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:36:26.229389 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82915f9e-582e-4684-8804-e6228f4680b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee992b33bf394700b927a306907bdced)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:36:26.230030 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82915f9e-582e-4684-8804-e6228f4680b7 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:36:26.378264 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-82915f9e-582e-4684-8804-e6228f4680b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee992b33bf394700b927a306907bdced', 'request_id': 'req-82915f9e-582e-4684-8804-e6228f4680b7'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:36:26.384819 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-82915f9e-582e-4684-8804-e6228f4680b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee992b33bf394700b927a306907bdced', 'request_id': 'req-82915f9e-582e-4684-8804-e6228f4680b7'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:36:26.491880 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-82915f9e-582e-4684-8804-e6228f4680b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee992b33bf394700b927a306907bdced', 'request_id': 'req-82915f9e-582e-4684-8804-e6228f4680b7'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:36:26.493336 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:36:26.505686 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-82915f9e-582e-4684-8804-e6228f4680b7 admin admin] The token cache is being invalidate because project ee992b33bf394700b927a306907bdced was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:36:26.510956 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-82915f9e-582e-4684-8804-e6228f4680b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:36:26.510956 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-82915f9e-582e-4684-8804-e6228f4680b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:36:26.510956 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1460/2880] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:36:26 2022] DELETE /identity/v3/projects/ee992b33bf394700b927a306907bdced => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:36:31.492267 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4e7149-b9e3-4137-9ada-2943ae743198 service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:36:31.493709 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-4f4e7149-b9e3-4137-9ada-2943ae743198 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:36:31.508175 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4e7149-b9e3-4137-9ada-2943ae743198 service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:36:31.564564 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04b30a9-a07a-4648-9f44-f0201f8d4d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:36:31.565135 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b30a9-a07a-4648-9f44-f0201f8d4d2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:36:31.565416 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b30a9-a07a-4648-9f44-f0201f8d4d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:36:31.565692 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b30a9-a07a-4648-9f44-f0201f8d4d2e admin admin] PATH_INFO: `/v3/users/2fc55eab88954503807d531cadd08543` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:36:31.566833 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04b30a9-a07a-4648-9f44-f0201f8d4d2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fc55eab88954503807d531cadd08543)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:36:31.568791 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04b30a9-a07a-4648-9f44-f0201f8d4d2e admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:36:31.795970 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-f04b30a9-a07a-4648-9f44-f0201f8d4d2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fc55eab88954503807d531cadd08543'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:36:31.808207 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-f04b30a9-a07a-4648-9f44-f0201f8d4d2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fc55eab88954503807d531cadd08543'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:36:31.808872 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-f04b30a9-a07a-4648-9f44-f0201f8d4d2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fc55eab88954503807d531cadd08543'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:36:31.817787 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-f04b30a9-a07a-4648-9f44-f0201f8d4d2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fc55eab88954503807d531cadd08543'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:36:31.817787 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-f04b30a9-a07a-4648-9f44-f0201f8d4d2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fc55eab88954503807d531cadd08543'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:36:31.896307 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1421/2881] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:36:31 2022] DELETE /identity/v3/users/2fc55eab88954503807d531cadd08543 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:36:32.155650 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82915f9e-582e-4684-8804-e6228f4680b7 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:36:32.157660 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-82915f9e-582e-4684-8804-e6228f4680b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:36:32.171014 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82915f9e-582e-4684-8804-e6228f4680b7 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:36:32.231464 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4806458-79e0-4ff5-ac59-c6a419ea1afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:36:32.232147 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4806458-79e0-4ff5-ac59-c6a419ea1afe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:36:32.232428 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4806458-79e0-4ff5-ac59-c6a419ea1afe admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:36:32.232701 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4806458-79e0-4ff5-ac59-c6a419ea1afe admin admin] PATH_INFO: `/v3/projects/a7664cec57944fccbca1522095567ab8` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:36:32.233997 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4806458-79e0-4ff5-ac59-c6a419ea1afe admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7664cec57944fccbca1522095567ab8)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:36:32.236695 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4806458-79e0-4ff5-ac59-c6a419ea1afe admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:36:32.389075 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-e4806458-79e0-4ff5-ac59-c6a419ea1afe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7664cec57944fccbca1522095567ab8', 'request_id': 'req-e4806458-79e0-4ff5-ac59-c6a419ea1afe'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:36:32.394683 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-e4806458-79e0-4ff5-ac59-c6a419ea1afe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7664cec57944fccbca1522095567ab8', 'request_id': 'req-e4806458-79e0-4ff5-ac59-c6a419ea1afe'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:36:32.473863 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-e4806458-79e0-4ff5-ac59-c6a419ea1afe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7664cec57944fccbca1522095567ab8', 'request_id': 'req-e4806458-79e0-4ff5-ac59-c6a419ea1afe'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:36:32.485999 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-e4806458-79e0-4ff5-ac59-c6a419ea1afe admin admin] The token cache is being invalidate because project a7664cec57944fccbca1522095567ab8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:36:32.486286 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-e4806458-79e0-4ff5-ac59-c6a419ea1afe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:36:32.486917 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-e4806458-79e0-4ff5-ac59-c6a419ea1afe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:36:32.488464 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1461/2882] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:36:32 2022] DELETE /identity/v3/projects/a7664cec57944fccbca1522095567ab8 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:36:32.492494 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04b30a9-a07a-4648-9f44-f0201f8d4d2e admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:36:32.494009 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-f04b30a9-a07a-4648-9f44-f0201f8d4d2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:36:32.507409 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04b30a9-a07a-4648-9f44-f0201f8d4d2e admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:36:32.516390 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:36:32.564751 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef75d0e4-7e08-42a7-9f0e-7283d4d776cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:36:32.565385 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef75d0e4-7e08-42a7-9f0e-7283d4d776cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:36:32.565664 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef75d0e4-7e08-42a7-9f0e-7283d4d776cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:36:32.565934 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef75d0e4-7e08-42a7-9f0e-7283d4d776cc admin admin] PATH_INFO: `/v3/users/d15ea8aac7b747a788d153bfcc629ea6` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:36:32.567072 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef75d0e4-7e08-42a7-9f0e-7283d4d776cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=d15ea8aac7b747a788d153bfcc629ea6)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:36:32.569074 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef75d0e4-7e08-42a7-9f0e-7283d4d776cc admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:36:32.813504 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ef75d0e4-7e08-42a7-9f0e-7283d4d776cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd15ea8aac7b747a788d153bfcc629ea6'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:36:32.825648 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ef75d0e4-7e08-42a7-9f0e-7283d4d776cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd15ea8aac7b747a788d153bfcc629ea6'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:36:32.826319 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ef75d0e4-7e08-42a7-9f0e-7283d4d776cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd15ea8aac7b747a788d153bfcc629ea6'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:36:32.833324 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ef75d0e4-7e08-42a7-9f0e-7283d4d776cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd15ea8aac7b747a788d153bfcc629ea6'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:36:32.833960 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ef75d0e4-7e08-42a7-9f0e-7283d4d776cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd15ea8aac7b747a788d153bfcc629ea6'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:36:32.936858 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1422/2883] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:36:32 2022] DELETE /identity/v3/users/d15ea8aac7b747a788d153bfcc629ea6 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:36:33.202330 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4806458-79e0-4ff5-ac59-c6a419ea1afe admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:36:33.205206 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-e4806458-79e0-4ff5-ac59-c6a419ea1afe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:36:33.218295 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4806458-79e0-4ff5-ac59-c6a419ea1afe admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:36:33.284903 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fceef7-607d-4d0b-adab-f531003e0faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:36:33.285553 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fceef7-607d-4d0b-adab-f531003e0faa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:36:33.285838 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fceef7-607d-4d0b-adab-f531003e0faa admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:36:33.286112 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fceef7-607d-4d0b-adab-f531003e0faa admin admin] PATH_INFO: `/v3/projects/05f83096d12840719d38b4bc6328dfc1` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:36:33.288770 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fceef7-607d-4d0b-adab-f531003e0faa admin admin] RBAC: Authorizing `identity:delete_project(project_id=05f83096d12840719d38b4bc6328dfc1)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:36:33.290735 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fceef7-607d-4d0b-adab-f531003e0faa admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:36:33.395756 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:36:33.412268 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-02fceef7-607d-4d0b-adab-f531003e0faa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05f83096d12840719d38b4bc6328dfc1', 'request_id': 'req-02fceef7-607d-4d0b-adab-f531003e0faa'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:36:33.417643 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-02fceef7-607d-4d0b-adab-f531003e0faa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05f83096d12840719d38b4bc6328dfc1', 'request_id': 'req-02fceef7-607d-4d0b-adab-f531003e0faa'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:36:33.473569 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-02fceef7-607d-4d0b-adab-f531003e0faa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05f83096d12840719d38b4bc6328dfc1', 'request_id': 'req-02fceef7-607d-4d0b-adab-f531003e0faa'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:36:33.484783 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-02fceef7-607d-4d0b-adab-f531003e0faa admin admin] The token cache is being invalidate because project 05f83096d12840719d38b4bc6328dfc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:36:33.485265 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-02fceef7-607d-4d0b-adab-f531003e0faa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:36:33.486351 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-02fceef7-607d-4d0b-adab-f531003e0faa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:36:33.488037 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1462/2884] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:36:33 2022] DELETE /identity/v3/projects/05f83096d12840719d38b4bc6328dfc1 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:36:42.933256 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=47,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:36:43.531254 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:DELETE=6,keystone:SELECT=14,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:37:27.593220 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef75d0e4-7e08-42a7-9f0e-7283d4d776cc admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:37:27.594642 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-ef75d0e4-7e08-42a7-9f0e-7283d4d776cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:37:27.650486 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef75d0e4-7e08-42a7-9f0e-7283d4d776cc admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:37:27.697125 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6515dfc8-ca5b-4017-ba86-dcf20ad1aded service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e31f2728056b460fa474478eb85ca1d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:37:27.697742 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6515dfc8-ca5b-4017-ba86-dcf20ad1aded service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:37:27.698026 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6515dfc8-ca5b-4017-ba86-dcf20ad1aded service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:37:27.698300 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6515dfc8-ca5b-4017-ba86-dcf20ad1aded service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:37:27.700716 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-6515dfc8-ca5b-4017-ba86-dcf20ad1aded service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:37:27.758587 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6515dfc8-ca5b-4017-ba86-dcf20ad1aded service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:37:27.760075 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6515dfc8-ca5b-4017-ba86-dcf20ad1aded service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:37:27.769829 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1423/2885] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:37:27 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:37:34.800622 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b17ca-5562-4403-abca-bc2c503510a3 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:37:34.801147 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b17ca-5562-4403-abca-bc2c503510a3 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:37:34.801147 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b17ca-5562-4403-abca-bc2c503510a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:37:34.820863 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-b03b17ca-5562-4403-abca-bc2c503510a3 None None] MFA Rules not processed for user `b665d491470f4a0b95c13f51e19d2046`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:37:34.822078 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-b03b17ca-5562-4403-abca-bc2c503510a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:37:34.829239 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1463/2886] 10.222.0.25 () {58 vars in 958 bytes} [Thu Aug 4 21:37:34 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:37:37.765947 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=68 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:37:44.816174 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:37:50.972095 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6515dfc8-ca5b-4017-ba86-dcf20ad1aded service cinder] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:37:50.973616 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-6515dfc8-ca5b-4017-ba86-dcf20ad1aded service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:37:51.047506 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6515dfc8-ca5b-4017-ba86-dcf20ad1aded service cinder] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:37:51.053366 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4911d6-931e-4ca7-bea1-29f642d540d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a2ce1df87fa47e5be301916bf3245fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:37:51.054115 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4911d6-931e-4ca7-bea1-29f642d540d8 service nova] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:37:51.054427 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4911d6-931e-4ca7-bea1-29f642d540d8 service nova] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:37:51.054705 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4911d6-931e-4ca7-bea1-29f642d540d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:37:51.056620 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-ed4911d6-931e-4ca7-bea1-29f642d540d8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:37:51.071581 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4911d6-931e-4ca7-bea1-29f642d540d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:37:51.073362 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4911d6-931e-4ca7-bea1-29f642d540d8 service nova] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:37:51.086813 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1424/2887] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:37:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:38:01.082623 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=34 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:38:19.889120 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03b17ca-5562-4403-abca-bc2c503510a3 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:38:19.890734 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-b03b17ca-5562-4403-abca-bc2c503510a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:38:19.963764 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03b17ca-5562-4403-abca-bc2c503510a3 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:38:19.968744 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8b1478-3b62-4b63-a00f-bbf4b23d132b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67d44a0098ab4c36b14b36e9e59ae75c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:38:19.969615 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8b1478-3b62-4b63-a00f-bbf4b23d132b service placement] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:38:19.970487 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8b1478-3b62-4b63-a00f-bbf4b23d132b service placement] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:38:19.970767 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8b1478-3b62-4b63-a00f-bbf4b23d132b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:38:19.972728 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-6f8b1478-3b62-4b63-a00f-bbf4b23d132b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:38:19.985551 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8b1478-3b62-4b63-a00f-bbf4b23d132b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:38:19.987128 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8b1478-3b62-4b63-a00f-bbf4b23d132b service placement] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:38:19.997515 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1464/2888] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:38:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:38:29.992908 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=32 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:39:43.621964 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4911d6-931e-4ca7-bea1-29f642d540d8 service nova] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:39:43.623467 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-ed4911d6-931e-4ca7-bea1-29f642d540d8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:39:43.682972 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4911d6-931e-4ca7-bea1-29f642d540d8 service nova] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:39:43.686563 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a75a40-8460-496d-821e-c6ab433f28e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08fcb91ee54344b5bb3870c585e25fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:39:43.687178 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a75a40-8460-496d-821e-c6ab433f28e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:39:43.687488 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a75a40-8460-496d-821e-c6ab433f28e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:39:43.687760 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a75a40-8460-496d-821e-c6ab433f28e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:39:43.689539 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-a1a75a40-8460-496d-821e-c6ab433f28e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:39:43.755625 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a75a40-8460-496d-821e-c6ab433f28e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:39:43.757007 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a75a40-8460-496d-821e-c6ab433f28e9 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:39:43.766739 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1425/2889] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:39:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:39:51.468522 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8b1478-3b62-4b63-a00f-bbf4b23d132b service placement] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:39:51.470000 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-6f8b1478-3b62-4b63-a00f-bbf4b23d132b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:39:51.484441 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8b1478-3b62-4b63-a00f-bbf4b23d132b service placement] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:39:51.489487 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d57f3b4-ee33-4cbe-bf3e-c5c91ed92c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:39:51.491467 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d57f3b4-ee33-4cbe-bf3e-c5c91ed92c3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:39:51.491467 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d57f3b4-ee33-4cbe-bf3e-c5c91ed92c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:39:51.491467 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d57f3b4-ee33-4cbe-bf3e-c5c91ed92c3d admin admin] PATH_INFO: `/v3/users/b665d491470f4a0b95c13f51e19d2046` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:39:51.491928 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d57f3b4-ee33-4cbe-bf3e-c5c91ed92c3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b665d491470f4a0b95c13f51e19d2046)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:39:51.493860 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d57f3b4-ee33-4cbe-bf3e-c5c91ed92c3d admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:39:51.745802 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-6d57f3b4-ee33-4cbe-bf3e-c5c91ed92c3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b665d491470f4a0b95c13f51e19d2046'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:39:51.758177 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-6d57f3b4-ee33-4cbe-bf3e-c5c91ed92c3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b665d491470f4a0b95c13f51e19d2046'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:39:51.758883 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-6d57f3b4-ee33-4cbe-bf3e-c5c91ed92c3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b665d491470f4a0b95c13f51e19d2046'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:39:51.759530 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-6d57f3b4-ee33-4cbe-bf3e-c5c91ed92c3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b665d491470f4a0b95c13f51e19d2046'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:39:51.867365 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-6d57f3b4-ee33-4cbe-bf3e-c5c91ed92c3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b665d491470f4a0b95c13f51e19d2046'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:39:51.880856 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1465/2890] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:39:51 2022] DELETE /identity/v3/users/b665d491470f4a0b95c13f51e19d2046 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:39:52.144552 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a75a40-8460-496d-821e-c6ab433f28e9 service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:39:52.156333 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a75a40-8460-496d-821e-c6ab433f28e9 service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:39:52.222846 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5304bf3c-fe49-4f48-bd9f-8cfe538fbcfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:39:52.222846 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5304bf3c-fe49-4f48-bd9f-8cfe538fbcfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:39:52.222846 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5304bf3c-fe49-4f48-bd9f-8cfe538fbcfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:39:52.222846 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5304bf3c-fe49-4f48-bd9f-8cfe538fbcfb admin admin] PATH_INFO: `/v3/projects/5b54066610f6483aa66bc27b5b2de6fe` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:39:52.223539 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5304bf3c-fe49-4f48-bd9f-8cfe538fbcfb admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b54066610f6483aa66bc27b5b2de6fe)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:39:52.224422 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5304bf3c-fe49-4f48-bd9f-8cfe538fbcfb admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:39:52.354195 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-5304bf3c-fe49-4f48-bd9f-8cfe538fbcfb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b54066610f6483aa66bc27b5b2de6fe', 'request_id': 'req-5304bf3c-fe49-4f48-bd9f-8cfe538fbcfb'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:39:52.356862 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:39:52.360450 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-5304bf3c-fe49-4f48-bd9f-8cfe538fbcfb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b54066610f6483aa66bc27b5b2de6fe', 'request_id': 'req-5304bf3c-fe49-4f48-bd9f-8cfe538fbcfb'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:39:52.449996 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-5304bf3c-fe49-4f48-bd9f-8cfe538fbcfb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b54066610f6483aa66bc27b5b2de6fe', 'request_id': 'req-5304bf3c-fe49-4f48-bd9f-8cfe538fbcfb'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:39:52.464139 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-5304bf3c-fe49-4f48-bd9f-8cfe538fbcfb admin admin] The token cache is being invalidate because project 5b54066610f6483aa66bc27b5b2de6fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:39:52.464432 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-5304bf3c-fe49-4f48-bd9f-8cfe538fbcfb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:39:52.465097 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-5304bf3c-fe49-4f48-bd9f-8cfe538fbcfb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:39:52.466657 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1426/2891] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:39:52 2022] DELETE /identity/v3/projects/5b54066610f6483aa66bc27b5b2de6fe => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:39:52.470712 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d57f3b4-ee33-4cbe-bf3e-c5c91ed92c3d admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:39:52.472438 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-6d57f3b4-ee33-4cbe-bf3e-c5c91ed92c3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:39:52.485887 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d57f3b4-ee33-4cbe-bf3e-c5c91ed92c3d admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:39:52.545463 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2e4ce1-7a9a-4ca1-aef6-e32414c131a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:39:52.546112 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2e4ce1-7a9a-4ca1-aef6-e32414c131a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:39:52.546392 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2e4ce1-7a9a-4ca1-aef6-e32414c131a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:39:52.546664 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2e4ce1-7a9a-4ca1-aef6-e32414c131a5 admin admin] PATH_INFO: `/v3/users/93956198082e44189d090d4c5fb95b63` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:39:52.547899 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2e4ce1-7a9a-4ca1-aef6-e32414c131a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93956198082e44189d090d4c5fb95b63)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:39:52.549927 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2e4ce1-7a9a-4ca1-aef6-e32414c131a5 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:39:52.760761 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-5e2e4ce1-7a9a-4ca1-aef6-e32414c131a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93956198082e44189d090d4c5fb95b63'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:39:52.773689 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-5e2e4ce1-7a9a-4ca1-aef6-e32414c131a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93956198082e44189d090d4c5fb95b63'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:39:52.774458 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-5e2e4ce1-7a9a-4ca1-aef6-e32414c131a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93956198082e44189d090d4c5fb95b63'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:39:52.775081 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-5e2e4ce1-7a9a-4ca1-aef6-e32414c131a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93956198082e44189d090d4c5fb95b63'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:39:52.867602 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-5e2e4ce1-7a9a-4ca1-aef6-e32414c131a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93956198082e44189d090d4c5fb95b63'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:39:52.870121 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:39:52.881156 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1466/2892] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Aug 4 21:39:52 2022] DELETE /identity/v3/users/93956198082e44189d090d4c5fb95b63 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:39:53.158458 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5304bf3c-fe49-4f48-bd9f-8cfe538fbcfb admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:39:53.159979 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-5304bf3c-fe49-4f48-bd9f-8cfe538fbcfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:39:53.176378 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5304bf3c-fe49-4f48-bd9f-8cfe538fbcfb admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:39:53.245637 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988a5675-f8c5-4e4c-945b-1e874dc80907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6bad33dfb24427b1ccf30de722b45a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6da47929012341928c673a2e0d1e97f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:39:53.246361 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988a5675-f8c5-4e4c-945b-1e874dc80907 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:39:53.246583 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988a5675-f8c5-4e4c-945b-1e874dc80907 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:39:53.246858 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988a5675-f8c5-4e4c-945b-1e874dc80907 admin admin] PATH_INFO: `/v3/projects/40498e1369524be3b9ef33e69f3539bd` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:39:53.248053 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988a5675-f8c5-4e4c-945b-1e874dc80907 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40498e1369524be3b9ef33e69f3539bd)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:39:53.250163 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988a5675-f8c5-4e4c-945b-1e874dc80907 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:39:53.373260 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-988a5675-f8c5-4e4c-945b-1e874dc80907 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40498e1369524be3b9ef33e69f3539bd', 'request_id': 'req-988a5675-f8c5-4e4c-945b-1e874dc80907'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:39:53.379286 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-988a5675-f8c5-4e4c-945b-1e874dc80907 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40498e1369524be3b9ef33e69f3539bd', 'request_id': 'req-988a5675-f8c5-4e4c-945b-1e874dc80907'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:39:53.461501 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-988a5675-f8c5-4e4c-945b-1e874dc80907 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40498e1369524be3b9ef33e69f3539bd', 'request_id': 'req-988a5675-f8c5-4e4c-945b-1e874dc80907'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:39:53.474331 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-988a5675-f8c5-4e4c-945b-1e874dc80907 admin admin] The token cache is being invalidate because project 40498e1369524be3b9ef33e69f3539bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:39:53.474619 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-988a5675-f8c5-4e4c-945b-1e874dc80907 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:39:53.475257 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-988a5675-f8c5-4e4c-945b-1e874dc80907 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:39:53.477132 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1427/2893] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Aug 4 21:39:53 2022] DELETE /identity/v3/projects/40498e1369524be3b9ef33e69f3539bd => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:40:03.047207 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=4 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:40:03.470480 dv-cinder-852208-1-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=57,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:43:27.268581 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2e4ce1-7a9a-4ca1-aef6-e32414c131a5 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:43:27.270026 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-5e2e4ce1-7a9a-4ca1-aef6-e32414c131a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:43:27.426261 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2e4ce1-7a9a-4ca1-aef6-e32414c131a5 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:43:27.493959 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ae451f-753b-45cd-9c89-7f182c95acf5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67d44a0098ab4c36b14b36e9e59ae75c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17dc3d39e2c048ca9070588f5717a578', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:43:27.494575 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae451f-753b-45cd-9c89-7f182c95acf5 service placement] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:27.494855 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae451f-753b-45cd-9c89-7f182c95acf5 service placement] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:27.495127 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae451f-753b-45cd-9c89-7f182c95acf5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:27.496966 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-70ae451f-753b-45cd-9c89-7f182c95acf5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:43:27.510627 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ae451f-753b-45cd-9c89-7f182c95acf5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:43:27.512265 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ae451f-753b-45cd-9c89-7f182c95acf5 service placement] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:43:27.524095 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1467/2894] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Aug 4 21:43:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:43:37.518515 dv-cinder-852208-1-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=89 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}